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Analysis  of  Projections  of  the  Transfer  Matrix 

in  2D  Ising  Models* 

Wee-Liang  Heng 

Abstract 

The  Ising  model,  originally  proposed  to  explain  properties  of  ferromagnets,  consists  of  a 
regular  lattice  whose  vertices  are  considered  to  be  ‘sites’  that  can  be  in  exactly  one  of  two 
possible  states.  Of  interest  is  the  partition  function,  which  is  the  sum  of  the  energ}'  of 
the  lattice  over  all  possible  configurations.  There  are  two  main  approaches  to  computing 
the  partition  function:  the  combinatorial  method  uses  an  expansion  whose  coefficients  are 
the  number  of  subgraphs  satsifying  certain  criteria;  the  algebraic  approach  introduces  a 
transfer  matrix  whose  spectral  radius  is  the  partition  function  per  spin.  In  the  semi-infinite 
2D  model  with  n  rows,  the  associated  transfer  matrix  M„  is  duodiagonal  of  order  2".  This 
thesis  introduces  a  special  class  of  subspaces  for  approximating  the  dominant  eigenvectors 
of  M„,  and  analyzes  the  projections  of  M„  and  its  adjoint  onto  these  subspaces.  We  shall 
show  that  the  projections  are  sparse  (with  2  or  4  nonzero  entries  per  column),  and  are  of 
order  0{n^2‘~^ )  where  /  is  a  parameter  of  the  subspaces.  Some  optimal  properties  of  these 
subspaces  are  established. 

‘The  author  gratefully  acknowledges  partial  support  from  ONR  contact  No.  N00014-90-J-1372. 
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Chapter  1 


Introduction 


1.1  Summary 


The  Ising  model  was  proposed  to  explain  properties  of  ferromagnets  but  since  then 
it  has  found  application  to  topics  in  Chemistry  and  Biology  as  well  as  in  Physics.  The  model 
arose  in  Statistical  Mechanics  and  consists  of  a  regular  grid  whose  vertices  are  considered  to 
be  ‘sites’  that  can  be  in  exactly  one  of  two  possible  states.  In  the  original  version  [Isi25]  each 
site  held  an  orientable  particle  that  could  have  its  spin  p  parallel  to  an  external  magnetic 
field  {n  =  +1)  or  antipaxallel  (/i  =  -1)  to  it.  Another  application  (the  formation  of  binary 
alloys)  has  /i  =  +1  if  the  site  contains  an  atom  of  type  A  and  /i  =  —1  if  it  contains  an 
atom  of  type  B.  In  studying  gases  n  =  +1  if  a  site  is  occupied  by  a  molecule  or  /i  =  0  if 
:t  is  empty.  An  excellent  introduction  to  the  Ising  model  targeted  at  a  general  audience  is 
[Cip87]. 

Early  work  focussed  on  ID  lattices  but  the  subject  really  came  to  life  in  1944 
when  Onsager  [Ons44]  derived  an  exact  closed  form  expression  for  the  partition  .unction 
(see  below)  for  an  infinite  2D  grid  with  no  external  magnetic  field.  This  expression  exhibited 
the  desired  singularity  that  signals  a  critical  temperature  Tc  at  which  a  p  uase  transition 
occurs.  Specifically  the  residual  magnetization  Mo(T)  that  remains  when  the  external 
magnetic  field  is  turned  off  is  positive  and  decreases  steadily  to  zero  as  T  -+  Tc  from  below 
but  simply  vanishes  for  all  T  >  Tc- 

Exact  solutions  for  nonzero  magnetic  fields  have  not  been  found  so  far  and  a 
number  of  researchers  have  turned  to  approximations.  There  are  tw’o  main  approaches. 

The  combinatorial  method  uses  an  expansion  of  Z^^,  the  partition  function  for  N 
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sites,  that  involves  for  its  r‘**  term  the  total  number  of  subgraphs  in  an  A^-node  graph  with 
exactly  r  edges  subject  to  certain  constraints.  Considerable  effort  has  gone  into  counting 
these  graphs  but  we  shall  say  no  more  on  this  topic.  See  {Kac68]  for  further  discussion. 

The  algebraic,  or  matrix  method  is  based  on  the  creation  of  a  transfer  matrix 
whose  spectral  radius  (the  largest  magnitude  among  the  eigenvalues)  yields  the  partition 
function  per  spin  [KW41].  This  study  is  concerned  with  projecting  the  transfer  matrix  onto 
a  class  of  special  subspaces  but  first  we  introduce  the  partition  function  and  the  related 
transfer  matrix. 

Suppose  that  the  grid  contains  N  sites  and  is  subject  to  an  external  magnetic 
field  of  strength  B.  The  interaction  energy  associated  w'ith  a  spin  configuration  jj.  = 
(/io,  •  •  •  is  defined  by 

E{^i)  =  -qBY^h,. 

neighbors 

Here  each  /z,  =  ±1,  J  is  the  coupling  constant  giving  the  strength  of  the  spin-spin  inter¬ 
actions  and  g  is  the  magnetic  moment  of  each  spin.  Usually,  neighbors  is  interpreted  as 
nearest  neighbors  but  broader  definitions  are  possible. 

The  “partition  function  per  spin”  at  temperature  T  is  defined  by 

z(J,B,T)  =  [  ^  ^-EM/kT^l/N 

aJl 

configurations 

for  an  A' -site  grid  and  k  is  Boltzmann’s  constant.  Several  quantities  of  physical  interest 
can  be  expressed  in  terms  of  z.  By  Boltzmann’s  law  is  the  probability 

of  occurence  of  configuration  /z  at  temperature  T.  The  free  energy  per  lattice  site  at 
temperature  T  is  -kTlogz  and  the  magnetization  per  spin  is  m  =  kT ^logr  [Tho79]. 

The  power  of  the  algebraic  approach  comes  from  the  introduction  of  a  matrix 
whose  dominant  eigenvalue  is  exactly  z„(J,B,T)  for  a  particular  semi-infinite  lattice  with 
n  rows.  There  are  several  matrices  that  can  be  associated  with  the  lattice,  some  symmetric, 
others  not.  We  use  the  one  with  the  fewest  nonzero  entries,  the  duodiagonal  matrix  Mn  of 
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order  2"  [Gar83].  We  illustrate  it  for  n  =  3  and  n  =  4: 


where  (with  appropriate  normalizations) 

a  =  6  =  and  c  = 

Since  M„  acts  on  vectors  in  R^"  by  multiplication  we  need  to  index  the  2"  positions  in  a 
vector  and  we  choose  a  standard  mechanism  of  using  binary  numbers.  We  give  an  example 
for  n  =  5: 

(11010)  2“*  +  2^  +  2*  =  26. 
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Note  that  the  n-bit  strings  here  have  no  direct  relation  to  the  spin  configurations  mentioned 
above. 

The  attractive  property  of  Af„  is  that  it  is  a  nonnegative  irreducible  matrix  whose 
dominant  eigenvalue  (called  the  Perron  root)  is  the  wanted  partition  function  per  spin.  Thus 
it  is  only  necessary  to  approximate  this  eigenvalue  to  the  desired  accuracy.  However  the 
second  eigenvalue  and  associated  eigenvectors  are  also  useful.  Moreover  A/„  is  exceedingly 
sparse;  it  has  exactly  2  non-zero  entries  per  row  (and  column)  arranged  in  a  regular  pattern. 
There  is  only  one  difficulty:  M„  is  of  order  2"  and  we  are  interested  in  the  case  n  — ►  oo. 
We  know  of  no  calculations  w’ith  n  >  20  at  present. 

We  note  that  the  difficulty  lies  not  in  A/„  but  in  the  the  representation  of  vectors  in 
R^".  Indeed  the  special  structure  of  A/„  permits  evaluation  of  M„v  for  any  2"-dimensional 
vector  V  with  great  efficiency.  Th  's  A/„  should  be  thought  of,  not  as  a  matrix  but  as  a 
linear  operator  that  requires  only  0(1)  data  for  its  definition,  i.e.  a  constant  amount  of 
space  independent  of  n. 

Sparse  vectors  occur  in  sparse  matrix  work  and  N.  Fuchs  [Fuc89],  when  applying 
the  Power  Method  to  A/„,  keeps  only  the  largest  1000  entries  of  each  vector.  This  device 
is  satisfactory  deep  within  the  ferromagnetic  region  of  the  model.  However  after  studying 
the  Perron  vector  in  cases  near  the  critical  temperature  we  found  that  it  contained  almost 
no  small  entries. 

As  a  substitute  for  sparsity  we  propose  to  limit  the  number  of  distinct  values  that 
can  occur  among  a  vector’s  components.  In  technical  terms  we  select  a  finite  family  of 
indicial  subspaces  {C,},  each  of  which  is  spanned  by  an  orthogonal  basis  of  indicial  vectors 
such  that  the  nonzero  entries  of  each  Xi  are  ones,  and  the  positions  of  the 
nonzero  entries  of  i,  and  Xj,  i  ^  j,  are  disjoint.  We  then  approximate  the  top  two  column 
eigenvectors  of  A/„  from  a  member  of  {C,}.  The  top  two  row  eigenvectors  zu-e  approximated 
using  a  dual  family  {Tv,}. 

We  indicate  briefly  how  the  two  families  of  indicial  subspaces  can  be  used  to  obtain 
a  minimal  representation  for  the  top  two  eigenvectors  of  A/„  [PH91]. 

Step  1.  Select  C,  and  Tv,  from  the  two  families.  Also  represent,  in  compact  form,  the 
orthogonal  projection  P  of  the  transfer  matrix  A/„  onto  the  subspace  C,.  In  addition 
represent  the  projection  Q  of  the  adjoint  matrix  A/*  onto  the  subspace  Tv,. 

Step  2.  Compute  the  two  largest  eigenvalues  and  the  associated  row  and  column  eigenvec- 
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tors  of  P  and  Q.  These  are,  in  a  sense,  the  best  approximations  from  the  given  pair 
of  indicial  subspaces  C,  and  TZi.  However  they  may  not  be  good  enough. 

Step  3.  Evaluate  residual  norms,  condition  numbers  and  associated  error  bounds  and  es¬ 
timates.  If  the  estimates  are  satisfactory  then  compute  the  required  properties  of  the 
model  and  stop.  Otherwise  return  to  Step  1  with  the  next  member  of  each  family. 

We  now  discuss  our  choice  of  indicial  subspaces  for  approximating  the  column 
eigenvectors  of  A/„.  In  particular,  we  describe  how  we  decide  which  entries  of  the  eigenvector 
should  be  forced  to  have  the  same  value. 

Consider  the  expression  for  the  energy  associated  with  a  particular  state  //  — 
(/ii, . . .  /ij  =  ±1,  in  a  ID  Ising  model  with  only  n  sites: 

n— 1  n 

E(#x)  =  -(J 

t=i  1=1 

Observe  that  in  the  first  sum,  —1  occurs  t  times  if  there  are  t  transitions  (adjacent  changes 
of  sign)  in  the  configuration  /x.  Thus 

n— 1 

Yi  M./i.+i  =  n-l-2t. 

i=l 

Similarly 

n 

=  —  (n  —  fr)  =  2fc-n 

1=1 

if  there  are  exactly  k  -l-l’s  in  Thus  two  states  fx  and  p.  contribute  equally  to  the  associated 
partition  function  if  they  have  the  same  number  of  -|-l’s  and  the  same  number  of  transitions. 

The  idea  of  using  k  and  t  can  be  applied  to  the  2D  model.  However,  the  situation 
is  more  complicated  here  and  we  do  not  know  how  to  justify  the  special  role  of  k  and  t. 
Nevertheless,  numerical  evidence  does  indicate  the  utility  of  k  and  t. 

Table  1.1  shows  the  dominant  (Perron)  column  eigenvector  for  with  parameter 
settings  D  =  0.0001  and  T  =  2.2.  w’hich  is  within  4%  of  the  critical  temperature  Tc-  The 
eigenvector  has  been  normalized  to  have  largest  entry  1,  and  the  entries  of  the  table  have 
been  sorted  in  decreasing  value  of  the  components  of  the  eigenvector.  The  first  column  of 
the  table  shows  the  index  of  a  position  in  the  eigenvector,  wriiten  as  a  5-bit  binary  number 
(and  with  its  decimal  equivalent  in  parenthesis),  and  the  second  and  third  columns  give  the 
value  of  k  and  t  for  that  index.  The  last  column  gives  the  value  of  that  component  of  the 
eigenvector.  We  shall  write  the  indices  as  binary  numbers  in  the  following  discussion. 


position 

k 

t 

value 

mil  (31) 

5 

0 

1.0000000 

00000  (0) 

0 

0 

0.9967231 

10111  (23) 

4 

2 

0.5597178 

01000  (8) 

1 

2 

0.5583141 

11011  (27) 

4 

2 

0.5487801 

00100  (4) 

1 

2 

0.5473442 

11101  (29) 

4 

2 

0.5352361 

00010  (2) 

1 

2 

0.5337697 

11110  (30) 

4 

1 

0.5059892 

00001  (1) 

1 

1 

0.5045102 

10011  (19) 

3 

2 

0.4027520 

01100  (12) 

2 

2 

0.4022891 

11001  (25) 

3 

2 

0.3743792 

00110  (6) 

2 

2 

0.3738325 

OHIO  (14) 

3 

2 

0.3522495 

10001  (17) 

2 

2 

0.3521065 

10101  (21) 

3 

4 

0.3420300 

01010  (10) 

2 

4 

0.3415416 

11100  (28) 

3 

1 

0.3286180 

00011  (3) 

2 

1 

0.3279917 

01111  (15) 

4 

1 

0.3214520 

10000  (16) 

1 

1 

0.3209276 

10110  (22) 

3 

3 

0.3048252 

11010  (26) 

3 

3 

0.3044408 

01001  (9) 

2 

3 

0.3042764 

00101  (5) 

2 

3 

0.3038702 

11000  (24) 

2 

1 

0.2798594 

00111  (7) 

3 

1 

0.2797993 

10010  (18) 

2 

3 

0.2598551 

01101 (13) 

3 

3 

0.2598535 

10100  (20) 

2 

3 

0.2473428 

01011  (11) 

3 

0.2473127 

Table  1.1:  Perron  eigenvector  for  A/s  with  B  =  0.0001,  T 
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We  see  from  the  table  that  although  the  eigenvector  is  not  sparse,  those  positions 
with  the  same  values  of  k  and  t  have  component  values  that  are  quite  close.  For  example,  the 
positions  with  4  ones  and  2  bit  transitions  are  10111,  11011  and  11101,  and  their  component 
values  are  0.5597178,  0.5487801  and  0.5352361  respectively. 

A  careful  look  at  the  table,  however,  reveals  an  anomaly.  The  positions  00001  and 
10000  both  have  a  single  one  and  a  single  bit  transition,  but  their  values  are  very  different: 
0.5045102  and  0.3209276  respectively  (see  the  items  marked  t)-  The  same  problem  occurs 
with  positions  11110  and  01111  (items  marked  t).  This  anomaly  occurs  more  frequently 
for  higher  n’s,  and  shows  that  using  k  and  t  as  the  sole  criteria  for  forcing  equality  in 
eigenvector  components  is  inadequate. 

Observe,  however,  that  for  each  anomalous  pair,  the  two  positions  have  different 
trailing  bit.  This  suggests  using  the  trailing  bit  as  a  third  criterion.  More  generally,  we  can 
use  the  last  I  bits  of  each  index. 

With  each  value  of  /,  we  can  define  an  indicial  subspace  C/  C  R^"  for  which  x  £  Ci 
has  the  same  value  in  entries  i  and  j  if  the  binary  representations  of  i  and  j  have  the  same 
number  of  Is,  the  same  number  of  bit  transitions,  and  the  same  last  I  bits.  We  obtain  a 
dual  subspace  "RLi  for  approximating  the  row  eigenvectors  of  Afn  by  using  the  fiist  I  bits  of 
an  index  as  the  third  criterion. 

We  note  that  there  is  a  tradeoff  present  in  choosing  approximating  subspaces.  By 
forcing  equality,  we  reduce  the  dimension  of  the  subspaces  that  we  work  in,  but  at  the  same 
time,  we  sacrifice  the  accuracy  of  our  results.  A  discussion  of  the  approximating  properties 
of  our  indicial  subspaces  is  found  in  (PH91].  We  briefly  state  some  results  from  there. 

For  the  case  n  <  15,  we  were  able  to  compare  our  approximations  with  the  exact 
results.  In  our  first  implementation,  we  obtained  approximations  that  were  accurate  to 
7  decimal  places  for  n  =  14,  using  a  subspace  of  dimension  only  896  as  compared  to 
2^*  =  16384.  We  also  ran  the  code  for  n  >  20  (for  which  no  exact  resuBs  are  available),  and 
our  error  estimates  showed  that  the  subspaces  delivered  reasonable  approximations.  For 
example,  for  n  =  30,  the  appro.ximations  were  estimated  to  be  good  to  at  least  4  decimal 
places  using  a  subspace  of  dimension  as  small  as  5632  (note  that  2^*^  »  10®). 

We  now  give  an  overview  of  the  thesis. 

Chapter  2  formalizes  the  idea  of  indicial  subspaces  and  analyzes  the  structural 
properties  of  the  projections  of  Mn  onto  such  subspaces.  In  Section  2.1,  we  define  the 
building  blocks  of  indicial  .subspaces,  and  in  Section  2.2,  we  show  how  approximations  can 
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be  derived  from  these  subspaces.  We  then  study  the  action  of  A/„  on  our  indicia!  subspaces 
in  Section  2.3,  and  this  enables  us  to  analyze  the  structure  of  the  projections  of  Af„  in 
Sections  2.4  and  2.5.  We  shall  show  that  in  general,  the  projections  of  Mn  are  also  sparse 
with  either  2  or  4  nonzeros  per  column.  Finally,  in  Sections  2.6  and  2.7,  we  introduce  a 
more  general  class  of  indicia!  subspaces,  and  show  that  the  ones  w’e  have  used  are  the  best 
in  the  class. 

Chapter  3  is  concerned  with  the  combinatorics  of  indicia!  subspaces.  W'e  begin  by 
analyzing  a  restricted  class  of  indicia!  subspaces  in  Section  3.1,  where  we  will  determine  the 
dimension  of  those  subspaces  and  the  maximum  number  of  components  that  are  forced  to 
be  equal.  We  then  extend  the  results  to  arbitrary  indicia!  subspaces  in  Sections  3.2  and  3.3. 
The  key  result  in  that  chapter  is  that  our  indicia!  subspaces  have  dimension  0{n^2}  ^)- 

Appendix  A  considers  how  we  can  extend  the  results  of  Chapter  3  to  the  more 
genera!  class  of  indicia!  subspaces  introduced  in  Section  2.6. 


1.2  Basic  Notation  and  Terminology 

We  will  follow  Householder’s  conventions:  upper  case  Roman  letters  for  matrices, 
lower  case  letters  for  column  vectors,  and  lower  case  Greek  letters  for  scalars.  However,  the 
letters  i,  j,  k,  /,  m,  n  and  t  w'ill  be  reserved  for  integers.  All  matrices  and  vectors  will  be 
real.  The  transpose  of  A  will  be  denoted  by  A*,  and  the  inner  product  of  vectors  x  and  y 
by  (i,y)  =  x‘y.  We  will  exclusively  use  the  Euclidean  norm  for  vectors:  l|r||  =  \/x'x. 

As  the  theory  behind  our  indicia!  subspaces  is  intimately  connected  with  the  bi¬ 
nary  representations  of  numbers,  we  will  index  the  rows  and  columns  of  a  matrix,  and  the 
elements  of  a  vector,  starting  from  0,  unless  o'herwise  specified.  Thus  for  .4  G  and 

X  €  R"*.  ( A),,j  denotes  the  entry  in  row  i  and  column  j  of  A,  0  <  i  <  /  —  1,  0  <  j  <  m  —  1, 
and  x(  i)  denotes  the  t'*'  element  of  x,  0  <  i  <  m  —  1. 

The  I X  m  zero  and  identity  matrices  will  be  w'ritten  as  O/xm  and  Jixm  respectively; 
the  2"  X  2"  identity  matrix  will  be  written  as  J„.  For  Q  G  R**'"’.  we  let  span(Q)  denote  the 
subspace  of  R^  spanned  by  tin  n  columns  of  Q.  Similarly,  if  5  is  a  set  of  vectors  in  R^  the 
subspace  spanned  by  these  vectors  will  be  denoted  by  span(5). 

The  symbol  :=  will  denote  a  definition,  and  the  symbol  □  will  mark  the  end  of  a 

proof. 

By  a  (binary)  string  w'.  we  shall  mean  a  finite  sequence  of  Os  and  Is.  The  empty 
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string  is  denoted  by  e.  We  write  {0, 1}*  for  the  set  of  all  strings  (including  e),  and  {0, 1}  "  for 
the  set  of  n-bit  strings,  n  >  1.  The  length  of  a  string  u)  is  denoted  by  |u;|,  the  concatenation 
of  two  strings  u^i  and  uj2  by  oJi  ouJ2-,  and  the  reversal  of  a  string  uj  (i.e.  u  written  backwards) 
by 

eg.  {0,1}^  =  {000,001,010,011,100,101,110,111}, 

10011011  =  6,  010  o  110  =  010110,  001101^=101100. 

We  also  define  l^l  :=  0,  e''  :=  £,  and  £ou}  =  ljo£:=oj  for  any  string  lj. 

For  a  nonempty  string  u,  we  denote  its  bit  from  the  left  by  w{i),  2  =  1,2 . |u,  |. 

Thus  UJ  =  u;(l)  o  (_’(2)  o  •  ■  •  o  a;(ltjl),  and  uj^  =  a;(lu;l)  o  wdu^l  —  1)  o  •  •  •  o  u.’(l).  For  given  /. 
1  <  /  <  lu>|,  the  /-bit  prefix  of  uj  is  the  substring  a2(  1)  o  tj(2)  o  •  •  ■  o  u;(/),  and  the  /-bit  suffix 
of  UJ  is  the  substring  u;(lu;l  -/-|- 1)  o  u;(ltjl  -/-l- 2)  o  -  •  o  wdwl).  The  empty  string  £  will  be 
considered  to  be  the  0-bit  prefix  and  the  0-bit  suffix  of  any  string  uj.  We  shall  also  refer  to 
tj(l),  u;(l)ou;(2)  and  wd^l)  as  the  leading  bit,  leading  bit  pair  and  trailing  bit  respectively 
of  a  string  uj  with  Iw’l  >  2. 

There  is  a  natural  correspondence  between  binary  strings  and  the  nonnegative 
integers  N  arising  from  the  concept  of  the  binary’  representation  of  numbers.  We  formalize 
this  by  defining  two  functions: 

V  :  {O,  l}*  — *  N  mapping  uj  G  {O,  l}*  to  the  integer  value  it  represents  (^(e)  :=  0), 
and  for  n  >  1, 

cr„  :  {0, 1, . . . ,  2"  —  1}  — »  {0,  l}"  mapping  t  G  N  to  its  n-bit  binary  representation. 

We  note  that  there  is  no  uniqueness  in  these  maps:  two  different  strings  may  have  the  same 
value  under  v.  eg.  n(Oll)  =  2;(11)  =  3.  and  a  nonnegative  integer  is  mapped  to  different 
strings  under  different  <7„'s.  eg.  0-2(3)  =  11,  0-3(3)  =  011.  This,  however,  should  cause  no 
confusion.  We  can  extend  v  to  sets  of  strings:  r(E)  =  {2;(u;)  :  uj  G  E},  E  C  {0,  l}",n  >  1. 

Two  other  properties  of  binary  strings  that  are  of  interest  to  us  are  their  1-bit 
counts  and  bit  transition  counts. 

Definition  1.1  Let  uj  be  a  nonempty  string.  We  define  k(uj)  to  be  the  number  of  Is  ap¬ 
pearing  in  UJ.  and  t(uj)  to  be  the  number  of  bit  transitions  in  uj  (i.e.  0  to  1  or  1  to  0).  We 
also  define  k{£)  :=  0  and  r(e)  :=  0. 
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eg.  k(OOOOOO)  =  0,  ,^(111111)  =  6,  k(101101)  =  4, 

r(OOOOOO)  =  r(llllll)  =  0,  r(OOllll)  =  1,  r{l01101)  =  4,  r(OlOlOl)  =  5. 

We  note  that  for  u)  G  {0. 1}",  we  have  0  <  k(u;)  <  n  and  0  <  r(^J)  <  n  -  1. 

It  should  be  emphasized  that  the  functions  k  and  r  are  defined  for  binary  strings, 
and  not  for  the  binary  representation  of  numbers.  The  earlier  remarks  on  the  non-uniqueness 
of  binary  representations  apply  here.  In  particular,  the  bit  tr^msition  count  associated  with 
a  number  depends  on  the  choice  of  the  binary  representation.  Thus,  r(cT2(3))  =  7-{ll)  =  0 
but  r(cT3(3))  =  t(011)  =  1.  This  issue  should  be  kept  in  mind  in  any  implementation  of 
indicia!  subspaces. 
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Chapter  2 

Projections  of  Transfer  Matrices 
onto  Indicial  Subspaces 

2.1  Indicial  sets,  vectors  and  bases 

We  begin  by  defining  the  building  blocks  of  our  indicial  subspaces.  Each  such 
subspace  is  obtained  by  forcing  two  vector  components  to  have  the  same  value  if  the  bi¬ 
nary  representation  of  their  indices  have  the  same  number  of  I’s,  the  same  number  of  bit 
transitions  and  the  same  /-bit  suffixes,  where  /  is  a  fixed  nonnegative  integer.  By  grouping 
the  indices  of  equal-valued  components  together,  we  obtain  a  partition  of  the  collection  of 
indices,  and  an  associated  basis  for  the  subspace. 

We  first  illustrate  the  ideas  with  a  simple  example  where  no  suffixes  are  involved 
The  space  we  shall  work  in  is  R®  and  we  shall  regard  indices  as  3- bit  binary  strings.  Consider 
the  subspace  C  of  R®  obtained  by  forcing  two  components  to  have  the  same  value  if  their 
indices  have  the  same  1-bit  count  and  the  same  bit  transition  count.  It  is  easily  verified 
that  among  the  eight  3-bit  strings  000,  . . . ,  111,  there  axe  only  two  pairs  of  strings,  namely 
{001,100}  and  {011,110},  which  satisfy  the  above  criteria.  By  grouping  the  indices  of 
equal- valued  components  together,  we  have  the  following  partition  of  {000, . . .,  Ill}: 

{{000},  {001,100},  {010},  {011,110},  {101},  {111}}.  (2.1) 

We  call  each  member  of  the  partition  an  indicial  set.  To  each  indicial  set  I,  we  can  associate 
an  indicial  vector  x,  which  has  ones  in  positions  whose  indices  are  in  I,  and  zeros  elsewhere. 
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For  the  partition  (2.1),  we  have  the  following  indicia!  vectors: 

xj  =  (1000000  0)',  X2  =  (0  1  0  0  1  0  0  0)*,  13  =  (0  010000  0)’, 
i4  =  (0  0  0  1  0  0  1  0)*,  i5  =  (0  0  0  0  0  1  0  0)’,  X6  =  (0  000000  1)*}. 

A  moment’s  thought  will  reveal  that  {xi, . .  .,X6}  is  a  basis  for  the  subspace  C. 

Formally,  we  define 

Definition  2.1  Let  n>l,0<k<n,0<t<n  —  1,  and  oj  6  {0,  l}’  with  |u;|  <  n.  Define 

■—  ^  {Oil}"  •  ^(m)  =  anduj  is  the  |u>|-bit  suffix  of  fx). 

I"  ,  is  called  a  suffix-based  indicia!  set.  For  a  given  lj  £  {0, 1}*  with  |u;|  <  n,  we  call  a 
pair  k,  t  admissible  if  I"  t  ^  ®- 


eg-  Ilo.o  =  {0000},  It  3  2 


{1011,1101},  1^12,2  =  {1001}. 


We  leave  it  as  an  exercise  for  the  reader  to  verify  that  Ij  3  3  =  0- 


Definition  2.2  Suppose  E  C  {0,  l}",  n  >  1.  Define  the  vector  xg  6  by: 


XE(i)  ;= 


1  t/<T„(t)GE 

0  t/«r„(i)^E 


0  <  i  <  2"  -  1. 


xg  can  be  regarded  as  the  tabulation  of  the  characteristic  function  x-£  of  E.  If  "E  is  a  suffix- 
based  indicial  set,  i.e.  E  =  some  w,  k  and  t,  we  call  xg  a  suffix-based  indicia! 

vector  and  we  also  write  it  as  xf^  f.  Note  that  x^  =  0,  and  that  1|xe||^  =  |E|. 


Definition  2.3  Let  n  >  1,  0  <  I  <  n.  Define 


^n.l  ■=  {^Z.k.t  :  kl  =  ^  admissible). 
We  call  S„j  an  indicial  basis  and  span  fS„jJ  an  indicial  subspace. 


We  rework  the  illustrative  example  using  our  new  terminology: 

eg-  I^.o.o  =  {000},  I?  ,  ,  =  {001,100},  ,  2  =  {OlO}, 

132.,  =  {011,110},  132.2  =  {101},  1330=  {111}, 

53.0  =  {(1000000  0)*,  (0  1  0  0  1  0  0  0)*.  (0  0  1  0  0  0  0  0)*, 

(0  0  0  1  0  0  1  0)*,  (0  0  0  0  0  1  0  0)*,  (0  0  0  0  0  0  0  1)*}. 
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Central  to  our  analysis  of  the  action  of  the  transfer  matrix  on  indicial  bases  (see 
Section  2.3)  are  the  following  two  types  of  operators  on  sets  of  binarj'  strings.  The  first  type 
will  enable  us  to  relate  the  image  of  an  indicial  vector  under  the  transfer  matrix  to  another 
indicial  vector;  the  second  type  plays  a  role  in  the  decoupling  of  the  action  of  the  transfer 
matrix.  We  urge  the  reader  to  read  with  care  the  definitions  of  2^  and  2ijJ  +  1  below. 

Definition  2.4  For  a  nonempty  string  u,  the  strings  2^  and  2u^  +  1  are  defined  by: 

2u;  :=  u)(2)  o  u;(3)  o  •  •  ■  o  Lj(lu;|)  o  0 
and  2u>  +  1  :=  u;(2)  o  u!(3)  o  •  •  •  o  juji)  o  1. 

It  is  easily  verified  that  v(2^)  =  2v{oj)  mod  2l“'l  and  v{2ll)  +  1)  =  (2i’(u’)  +  1)  mod  2l'^l.  We 
extend  the  definition  to  sets  of  strings:  for  E  C  {0,  l}",  n  >  1, 

2E  :=  {2u;  ;  w  6  E}  and  2E  +  1  :=  {2u;  +  1  :  €  E). 

The  reader  fanailiai  with  binary  arithmetic  will  recall  that  multiplying  a  number  u.-  by  2 
corresponds  to  shifting  in  a  0  from  the  right,  and  that  multiplying  u)  by  2  and  then  adding 
1  corresponds  to  shifting  in  a  1.  The  strings  2u;  and  2u>  +  1  are  the  results  we  would  obtain 
when  we  perform  the  operations  in  a  computer  with  |u)l-bit  words. 

Definition  2.5  Let  E  C  {O.  l}",  n  >  2.  We  define 

E°  :=  {a;€E:a;(2)  =  0} 
and  E^  :=  {u;  €  E  :  u;(2)  =  l}, 

i.e.  E®  and  E^  are  subsets  of  strings  in  E  having  2"*^  leading  bit  0  and  1 
follows  from  the  definition  that  E  ts  the  disjoint  union  ofEP  and  E*.  //E 
write  E°  =  and  E^  =  ,. 

eg.  E  =  {0011.0100,0101,1000},  E°  =  {0011, 1000},  E^  =  {0100. 0101}. 
£={0010.0011},  E°  =  E.  E^  =  0. 

E  =  1^33  =  {0001011.0001101.0010011,0011001,0100011.0110001}, 

E°  =  {0001011,0001101,0010011,0011001},  E^  =  {0100011. OllOOOl}, 


respectively.  It 

—  k,t  • 
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2E°  =  {0010110,0011010,0100110,0110010}  =  IJo,3.4i 
2E°  +  1  =  {0010111,0011011,0100111,0110011}  =  IJi.4,3, 

2E^  =  {1000110,1100010}  =  IJo,3.3. 

2E^  +  1  =  {1000111, 1100011}  = 

As  the  last  example  indicates,  2E°,  2E°  +  1,  2E^  and  2E^  +  1  are  pairwise  disjoint  and  are 
themselves  suffix-based  indicia!  sets,  and  |2E®|  =  |2E°  +  l|  =  |E°|  and  |2E^|  =  |2E^  +  1|  = 
1E^|.  This  is  true  in  general  (see  Proposition  2.1.6  and  Theorems  2.1.7  and  2.1.8  below). 

In  discussions  where  the  value  of  n  is  assumed  fixed,  we  will  omit  it  in  writing 
indicia!  objects.  Thus,  we  write  instead  of  I"  t,,, 

and  t  respectively. 

In  an  analogous  fashion,  we  can  define  prefix-based  indicia!  sets  vectors 

j/"  f.  f  and  bases  T„  i.  It  turns  out,  however,  that  prefix-based  indicia!  objects  can  be  derived 
from  corresponding  suffix-based  ones.  This  will  be  explored  in  Section  2.4. 

We  note  here  for  future  analyses  some  crucial  properties  of  suffix-based  indicia! 
sets,  vectors  and  bases.  We  urge  the  reader  to  go  through  them  carefully,  with  the  exception 
of  the  proof  of  Theorem  2.1.8,  where  parts  (b),  (c)  and  (d)  may  be  skipped. 

Fundamental  Properties 

Proposition  2.1.1  For  fixed  n  and  |u;|,  the  nonempty  indicial  sets  disjoint.  Thus 

for  0  <  1  <  n,  Snj  is  an  orthogonal  basis  (i.e.  for  x,y  G  5„,/,  x  ^  y  ^  x‘y  =  0)  and  so  is 
linearly  independent. 

Proposition  2.1.2  For  fixed  n  and  |u;|,  the  collection  of  nonempty  indicial  sets  ,  ts  a 
partition  of  {0, 1}".  Thus  for  each  0  <  i  <  2"  -  1,  there  is  a  unique  x  6  5„,/  uiith  x(i)  =  1. 

Proposition  2.1.3  The  trailing  bit  and  the  parity  of  the  transition  count  of  a  nonempty 
string  p.  determines  its  leading  bit. 

Proof.  An  even  number  of  transitions  (viewing  from  the  right  end  of  p  to  its  left  end) 
restores  its  trailing  bit  whereas  an  odd  number  of  transitions  reverses  its  trailing  bit.  □ 

Proposition  2.1.4  The  strings  in  each  indicial  set  n  >  2.  ^  e,  all  have  the  same 

leading  bit  since: 
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(a)  the  trailing  bit  of  u  determines  the  trailing  bit  of  each  n  G  I"  ^ 

(b)  by  Proposition  2.1. S,  ift  is  even,  the  leading  bit  of  each  P  €  k  t  the  same 

as  its  trailing  bit;  ift  is  odd,  the  leading  bit  must  be  different. 

Proposition  2.1.5  The  strings  in  each  1"'°  n  >  3,  u;  ^  e,  all  have  the  same  leading  bit 
pair  and  the  same  trailing  bit  since  Proposition  2.1.4  determines  the  leading  and  trailing 
bit,  and  the  2"^*  leading  bit  is  0  by  definition.  Similarly,  the  strings  in  each  dr  ”  >  3, 
•jj  ^  e,  all  have  the  same  leading  bit  pair  and  the  same  trailing  bit. 

Proposition  2.1.6  Let  E  C  {0, 1}",  n  >  2.  The  sets  2E°,  2E°  +  1.  2E^  and  +  1  arc 
pairwise  disjoint  since  by  definition: 

(a)  strings  in  2E°  have  leading  bit  0  and  trailing  bit  0 

(b)  strings  in  2E°  +  1  have  leading  bit  0  and  trailing  bit  1 

(c)  strings  in  2E^  have  leading  bit  1  and  trailing  bit  0 

(d)  strings  in  2E^  +  1  have  leading  bit  1  and  trailing  bit  1 

Theorem  2.1.7  Let  E  =  n  >  2,  u  ^  e.  Then  |2E°|  =  |2E0  +  l|  =  |E°1  and 

|2E^|  =  |2Ei  +  1|  =  |Ei|. 

Proof.  Let  u)i,u>2  €  E®  with  2u\  =  2uJ2.  Then 

u;i(2)  o  u;i(3)  o  •  •  •  o  u;i(n)  o  0  =  u;2(2)  o  u;2(3)  o  ■  •  •  o  u>2(n)  o  0 


and  so 

=  u;i(l)  o  u;i(2)  o  •  •  •  o  u;i(n)  =  u;2(l)  o  0^2(2)  o  o  u;2(n)  =0^2 

since  uJi(l)  =  a>2(  1)  by  Proposition  2.1.4  (note  that  wi,u;2  6  E).  Therefore,  for  wi.u^2  €  E°. 

^  ijj2  =>  2a;i  ^  2uj2,  and  we  have  a  one-to-one  correspondence  between  E°  and  2E°.  So 
|2E0|  =  |E°|.  Similarly,  we  have  |2E0  -t-  l|  =  |E°|  and  |2Ei|  =  |2E^  4-  l|  =  \E^\.  □ 

Theorem  2.1.8  Let  E  =  n  >  3.  0  <  |u;|  <  n.  Then  2E°,  2E0  1.  2E^  and  2E^  -f-  1 

are  themselves  suffix- based  indicial  sets. 
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Proof.  There  are  four  cases  to  consider,  depending  on  the  trailing  bit  of  u  and  the  parity 
of  t.  We  will  prove  the  result  for  the  first  case;  the  proofs  for  the  remaining  three  cases  are 
similar. 

a)  trailing  bit  of  a;  is  0  and  t  is  even: 

We  claim  that  2E®  =  I"oO,it,r  Suppose  /i  €  E°  C  E.  Then  n(2)  =  0,  fi(n)  =  0, 
and  by  Proposition  2.1.4  applied  to  E,  ^(1)  =  0.  It  is  clear  that  n  and  2/x  have  the  same 
1-bit  count  and  the  same  transition  count,  and  that  a;  o  0  is  the  (|u>|  -f  l)-bit  suffix  of  2fi. 
So  2/i  G  IZo0.k.i^  and  2E0  C 

Conversely,  let  /i'  G  I"o0.it,r  Then  by  definition,  /i'(n)  =  0  and  /i'(n-l)  =  w(|w|)  = 
0.  By  Proposition  2.1.4  applied  to  I"o0.fc.e’  Thus  fi'  =  0ofi'{2)o-  ■  •o//'(n  — 2)oOoO. 

If  we  let  =  0o0ofi'(2)o-  ■  ■o^'(n  —  2)o0,  we  see  that  /i'  =  2/i.  It  is  clear  that  //  and  /x'  have 
the  same  1-bit  count  and  the  same  transition  count.  So  /x  G  and  ^  2E°. 

Therefore,  we  have  2E°  =  I"o0fc  «•  ‘^an  show  in  a  similar  manner  that: 

2E°  -f  1  =  I"oi,fc-n,t+i ’  ~  +  1  = 

b)  trailing  bit  of  w  is  0  and  t  is  odd: 

2E°  =  Ko0.k-u-i^  2E0+l  =  lZoi,k.n 
2E^  =  lZo0.k-i.r  +  1  = 

c)  trailing  bit  of  u;  is  1  and  t  is  even: 

2E^  =  IZo0.k-u^  2E^  +  l  =  lZol.k..-^^ 

2E^  =  ^wo0,fc-l,f+l  ’  +  1  =  VZol.k.f 

d)  trailing  bit  of  u;  is  1  and  t  is  odd: 

2E'=i:o0.u-  2Ei-hl  =  i:„i.*+,,_,.  □ 

2.2  Subspace  Approximations 

The  orthogonal  basis  S^j  was  defined  in  the  previous  section.  There  is  a  unique  or¬ 
thogonal  projection  of  Mn  onto  span(5„,/ )  and  our  method  takes  the  two  largest  eigenvalues 
of  this  projection  as  approximations  to  those  of  3/„, 


Here  are  the  details. 
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Let  X  6  be  a  matrix  whose  columns  axe  the  vectors  in  5„,/.  Let  A'*A'  = 

Dx,  a  positive  definite  diagonal  matrix  (with  integer  entries).  Let  A'  =  The 

projector  on  Sn,i  is  A'A'*,  and  the  projection  of  A/„  is  XX’M„XX',  and  its  representation 
in  the  basis  given  by  A'  is 

P^j  =  X-M„X  =  D-^'^X’MnXD-y}^. 

Observe  that  P^^  is  diagonally  similar  to 

PZ  =  D-^^X’M„X  =  D^^Pp^iDf. 

It  is  slightly  more  convenient  to  work  with  P^/  than  with  P^^. 

In  order  to  obtain  good  error  estimates  we  approximate  M  *  as  well  but  for  reasons 
discussed  in  Section  2.7  we  project  it  onto  the  subspace  Tnj  dual  to  S„j.  If  1'  is  a  matrix 
whose  columns  are  the  vectors  in  Tnj  we  compute  the  representation  of  the  orthogonal 
projection  of  A/*  as 

=  Dy^Y'M'Y. 

Let  xj  and  X2  be  the  largest  two  eigenvalues  of  In  the  absence  of  two  such 
positive  eigenvalues  we  abandon  the  basis  immediately  and  increase  /  by  1.  Thus  we 
compute  (x,,p,,/i*),  i  =  1,2, 

PZ9\=9\^\^  h\P^A  =  -^\h\,  P^j92  =  92T^2,  h^P^i  =  Tt2h\,  0<X2<Xi. 
Approximate  eigenelements  for  A/„  are  defined  as 

(x..A<7.,h*L>3^.'A'*),  t  =  l,2, 

although  we  may  not  wish  to  form  these  eigenvectors  until  we  are  confident  that  they  are 
satisfactory.  We  define  similar  approximations  for  M*  using  the  basis  Tn^. 


2.3  Action  of  duodiagonal  matrices  on  indicial  bases 

Our  primarj’  goal  is  to  analyze  the  structure  of  the  column  projection  matrix  P^^ 
and  of  the  row  projection  matrix  P^^.  This  requires  us  to  understand  the  action  of  the 
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transfer  matrix  M„  on  basis  vectors  x  in  Sn,i  for  /  >  1.  In  this  section,  we  shall  see  how 
to  decouple  the  action  of  M„,  and  thus  express  MnX  as  a  linear  combination  of  indicia! 
vectors.  Section  2.4  then  analyzes  the  structure  of  and  for  /  >  1.  The  degenerate 
case  I  =  0  will  be  discussed  in  Section  2.5. 

Let  n  be  a  fixed  integer  >  3,  and  /  be  a  fixed  integer  >  1.  Recall  that  P^i  = 
X' MnX  where  the  columns  of  A'  are  vectors  in  5„,/,  and  Dx  =  A'*A'.  To  elucidate 
the  action  of  Mn  on  A',  we  introduce  a  general  class  of  duodiagonal  matrices  of  which 
Mn  is  a  special  case.  We  first  illustrate  it  for  n  =  4: 


In  general,  we  consider  the  2"  x  2"  duodiagonal  matrix: 


where 


U(0) 
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= 


.2”-*x2'— * 


C7(i)  = 


0 

u}  0 

0  tjj 

0  uf 


t/p)  = 


*2"-‘x2"-2 


[/ (2)  = 


Uj  0 

ul  0 

0  U2 
0  U2 


t/P*  = 


which  we  shall  denote  by 


«o  “0 
«o  «o 


Uj  ^2 

«2  “2 


= 


0 

“3  0 

0 

0  ii§ 


«3  «3 

«3  «3 


and  abbreviate  by  £/„.  The  transfer  matrix  Mn  is  then  equal  to 


a  ’  a“' 


fr-‘  6-1 


6-1  ft-’ 


c”’  c”’ 


We  shall  denote  the  fc’**  column  of  U„  by  u^,  =  0, 1, . . . ,  2"  —  1. 

Efficient  processing  with  duodiagonal  matrices  U„  depends  on  the  following  key 
observations  regarding  their  nonzero  entries: 

(a)  the  nonzero  entries  of  Uk  occur  exactly  at  positions  2k  mod  2"  and  {2k  +  1)  mod  2". 

(b)  the  parameters  and  (i  =  0,1,  j  =  0, 1,2,3)  are  the  nonzero  entries  of 

*^2"-*j  +  2lt+i'  0  <  ^  <  2"  ^  —  1, 

eg.  for  C/5,  the  parameters  uj  and  (i.e.  i  =  1,  j  =  1)  are  the  nonzero 
entries  of  ug,  tin^  «i3  and  ujs. 

Equivalently,  the  parameters  and  are  the  nonzero  entries  of  Uk,  for  those 

fc  6  {0, 1,. .  .,2"  -  1}  where  the  leading  bit  pair  of  (T„{k)  is  (T2(j)  the  trailing  bit 
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of  Onik)  is  CT\{i).  In  the  above  example,  the  only  5-bit  strings  with  leading  bit  pair 
01  and  trailing  bit  1  are 

or5(9)  =  01001,  05(11)  =  01011,  05(13)  =  01101  and  05(15)  =  01111. 
Combining  the  above  observations,  we  have: 

Proposition  2.3.1  Fori  =  0, 1,  j  =  0, 1,2,3,  the  parameter  uj'  appearsas  the  (r(w),t’(2^)) 
entry  0/ f/„  where  uj  €  {0,l}"  satisfies  ou>{2)  =  02(i)  andu;(n)  =  Oi(t);  the  parameter 
appears  as  the  (n(u;),  v{2ijj  -f-  1))  entry  of  Un  for  those  same  u ’s. 

We  remind  the  reader  that  for  a  string  p  6  {0,1}",  v{p)  denotes  the  integer  value  it 
represents,  and  that  for  i  =  0,1,  j  =  0, 1,2,3,  Oi(i)  and  02(i)  denote  the  1-bit  binary 
representation  of  j  and  the  2-bit  binary  representation  of  j  respectively  (see  Section  1.2). 

Suppose  E  C  {0, 1}"  is  such  that  all  strings  in  E  have  the  same  leading  bit  pair 
and  the  same  trailing  bit.  Recalling  that  a  matrix- vector  product  is  a  linear  combination  of 
the  columns  of  the  matrix  with  coefficients  given  by  the  entries  of  the  vector,  we  see  that 

UnXE  =  Ufc  =  X]  “'t. 

k  where  k  where 

XE(fc)  =  1  w„(fc)  €  E 

From  observation  (b),  the  Ufc’s  involved  in  the  vector  sum  have  the  same  two  parameters 
Uj'  and  (for  some  i,  j)  as  their  nonzero  entries,  and  applying  observation  (a)  to  each 

such  Ufc,  we  see  that  the  nonzero  entries  of  U„xe  have  indices  in  i;(2E)  U  r(2E  +1);  uj' 
appears  at  indices  v(2E)  while  uj'"*'*  appears  at  indices  t;(2E-t-  1).  We  have  thus  shown  the 
following: 

Theorem  2.3.2  Let  E  C  {0,  l}",  n  >  3.  be  such  that  all  strings  in  E  have  the  same  leading 
bit  pair  and  the  same  trailing  bit.  Then 

U„XE  =  uj'x2E  +  uj‘+*i2E-n 

for  some  i.  j. 

Consider  now  applying  U„  to  the  indicial  vector  iw.fc.t  €  Let  E  =  l^,k.t- 

Recall  from  Definition  2.5  that  E°  =  1°  and  E^  =  (E°  and  E^  could  possibly  be 

empty),  and  that  E  is  the  disjoint  union  of  E°  and  E*.  By  Proposition  2.1.5,  E°  and  E^ 
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each  contain  strings  with  the  same  leading  bit  pair  and  the  same  trailing  bit.  Applying 
Therorem  2.3.2  to  E®  and  to  E^,  we  have; 

=  (“j'^2E0  +  +  (4‘’^2E1  +  “j'’^^'^2E1  +  i) 

for  some  0  <  i,i'  <  1  and  0  <  j,j'  <  3  with  (i,i)  ^  (^^  jO-  By  Theorem  2.1.8,  2E°,  2E°  +  1. 
2E^  and  2E^  +  1  are  themselves  suffix-based  indicia!  sets.  Enumerating  the  possibilities, 
we  have  the  following  four  cases  for  the  formation  of  f^na^E- 

Theorem  2.3.3  Let  x,^^ic.t  €  S„j  and  E  =  Then  UnXE  ^  o.  linear  combination  of 

(possibly  zero)  suffix- based  indicial  vectors  as  follows: 

^a)  trailing  bit  of  u  is  0  and  t  is  even: 

^n^E  ~  (^O^wo0,k,t  "b  .t-fl  )  "b  — 1  "b  ti  j  Xu;ol,fc+l,<  ) 

(b)  trailing  bit  of  u  is  0  and  t  is  odd: 

^n^E  ~  (tt^a^woO.fc  — 1,J-1  "b  ttjatwol.fc.t)  "b  (WaatwoO.Ar— l,t  "b  U3X^,ol,J:,t+l) 

(c)  trailing  bit  ofu^isl  and  t  is  even: 

^n^E  ~  ( ai^aTj^ioo.fc— 1,<  4  1 )  "b  (u3a;u»oo,fc— i.t-t- 1  4  ) 

(d)  trailing  bit  of  u;  is  1  and  t  is  odd: 

^n^E  ~  ( ^O^'*’o0.fc,t+1  4"  ^o^woi,k+l,t^  "b  (a^X^oQ  f-  i  U^I,^ol.fc+l,f  — 1  ) 

As  an  illustration  of  Theorem  2.3.3,  we  work  out  the  action  of  t/5  on  some  of  the 
basis  vectors  in  55,1  in  the  example  below.  The  columns  of  the  32  x  32  identity  matrix  I5 
will  be  denoted  by  {cq.  Ci. . . .  ,631}.  We  urge  the  reader  to  go  through  the  example  carefully, 
and  to  observe  in  each  case  how  the  result  of  applying  to  an  indicial  vector  could  be 
expressed  as  a  linear  combination  of  2  or  4  indicial  vectors. 

eg.  (a)  13.1,2  =  {00010,00100,01000},  ar3,i,2  =  ^2  +  €4  +  eg. 

—  ^5^2  4  ^'564  + 

=  U2  +  “4  4  «8 

=  (uge4  +  u^es)  4-  (uS^8  4  ujeg)  4  (u?ei6  4  ulcir) 
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=  Uq(€4  +  Cs)  +  +  ^9)  +  ^1^16  + 

=  ^^0^00.1, 2  +  “0^01,2,3  +  “1^00, 1.1  “1^01,2,2 

(b)  Iq  2,3  =  {  lOOlOt  10100}.  2^0  2,3  =  ^18  +  ^20. 

^5^0,2  1  ~  ^18  +  ^20 

=  (112^4  +  112^5  )  +  («2^8  +  “2^9) 

=  *^2^00, 1,2  +  ^22^01,2.3 

(^)  If, 4, 2  =  {10111,11011,11101},  Jrf42  =  f23  +  C27  +  e29. 

^"52:1,4,2  =  W23  +  U27  +  W29 

=  (U2^14  +  “2*^is)  +  (“3^22  *-'jf23)  +  (*^3^26  +  *^3^27) 

=  “2^10,3.2  +  ^22'li,4,l  +  “3-‘lO,3,3  +  ^3^11,4,2 
(rf)  If, 3, 1  =  {00111},  lf,3,,  =^7, 
f^'52:f,3,i  =  117 

=  ^^0^14  +  ^0^15 
=  Uo*fo,3,2  +  “02^11,4,1 

Returning  to  our  analysis,  we  note  that  if  E°  =  0,  then  Xjgo  =  ^2E0+i  ~ 
that  if  =  0,  then  Zjei  =  ^2Ei  +  i  ~  However,  E°  and  E^  cannot  both  be  empty  since 
E  =  E®  U  El  and  E  is  nonempty.  In  addition,  the  nonzero  vectors  eumong  I2EO.  ^2E0+1' 
and  X2£i^.j  are  distinct  from  each  other  since  2E®,  2E®  +  1,  2Ei  and  2Ei  +  1  are 
pairwise  disjoint  by  Proposition  2.1.6.  So  U„Xf^  is  a  linear  combination  of  either  2  or  4 
suffix- based  indicial  vectors. 

We  can  in  fact  relate  UnXf^  to  the  vectors  in  the  basis  5„,/.  Define  a  subvector  of 
a  vector  x  to  be  a  vector  obtained  by  setting  zero  or  more  entries  of  x  to  0,  i.e.  a  subvector 
has  the  same  number  of  entries  but  more  of  them  are  0;  analogously,  a  subcolumn  of  a 
matrix  ,4  is  a  column  obtained  by  setting  zero  or  more  entries  of  some  column  of  .4  to  0. 
Recall  that  E  =  I,*,,*:,^.  and  u  €  {O,  l}^  By  Theorem  2.1.8,  each  of  2E®  and  2Ei  is  of  the 
form  I„,o0.t',('  lor  some  k'  and  t',  and  each  of  2E®  +  1  and  2Ei  4-  1  is  of  the  form  lujoi.t",/" 
for  some  k"  and  t".  Now. 


u;  o  0  =  u;(l)  o  (u;(2)  o  •  •  •  o  u;(>')  o  0)  =  u;(  1)  o  2^;, 
and  a;ol=u;(l)o(w’(2)o--ou-'(/)ol)  =  u;(l)o(2a;+l). 
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Therefore  IwoO.fc'.t'  Q  l2u),fc',t'  and  Iwoi,ib'',«"  Q  since  2u;  and  2uj  +  1  are  suffixes 

of  w  o  0  zuid  u;  o  1  respectively.  It  follows  then  that  ^2E0+i’  ^2Et  ^2Et+i 

subvectors  of  vectors  in  S„j  since  |2u;|  =  |2u;+  l|  =  /.  Thus  UnX-£  is  a  linear  combination  of 
either  2  or  4  subvectors  in  S„j  (with  each  subvector  arising  from  a  different  vector  in  5„  ; ). 


2.4  Structure  of  the  column  and  row  projection  matrices 

Armed  with  our  understanding  of  the  action  of  duodiagonal  matrices  on  suffix- 
based  indicia!  bases,  the  analysis  of  the  structure  of  the  column  projection  matrix  P^i 
becomes  straightforward.  We  shall  show  that  for  /  >  1,  P^j  is  sparse  with  either  2  or  4 
nonzero  entries  per  column,  and  we  shall  precisely  locate  the  positions  of  those  entries,  and 
express  their  values  in  a  way  that  enables  them  to  be  computed  without  using  vectors  in 
R^".  By  exploiting  an  important  relationship  between  suffix- based  and  prefix-based  indicial 
objects,  we  shall  show  that  the  row  projection  matrLx  P^^  has  the  same  sparsity  structure 
as  the  corresponding  column  projection  matrix  P^i- 

Recall  that  P^j  —  D'^^X‘M„X ,  where  the  columns  of  A'  are  the  vectors  in  5,,.; 
and  Dx  =  X“X.  We  index  the  rows  and  columns  of  P^j  by  the  triple  where 

is  a  column  of  A'.  Then  the  entry  in  row  and  column  {u),k,t)  of  P^[  is  given  by: 

II  '  112  “  IT  ,k’ ,t' 1  ^nXu),k,t)' 

Recall  that 


Let  E  =  The  analysis  of  Section  2.3  apphed  to  Af„  shows  that  A/rXe  is  a  linear 

combination  of  either  2  or  4  subcolumns  of  A'  (with  each  subcolumn  arising  from  a  different 
column  of  A').  Since  the  columns  of  A'  have  pairwise  disjoint  supports  (cf.  Proposition  2.1.2), 
pZi  is  sparse  with  2  or  4  nonzero  entries  per  column  arising  from  the  nonzero  inner  products 
Specifically,  column  {Lj,k,t)  P^j  has  2  nonzero  entries  if  one  of  E°  and 
E^  is  empty,  and  has  4  nonzero  entries  if  both  E°  and  E^  are  nonempty.  For  completeness, 
we  enumerate  the  possible  nonzero  inner  products  (cf.  Theorem  2.3.3).  The  reader  who  is 
not  interested  in  the  details  may  wish  to  skip  the  enumeration. 
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(a)  trailing  bit  of  w  is  0  and  t  is  even: 

=  allujo0,ik,il 

=  a|2E°| 

=  a|E°|  by  Theorem  2.1 .7. 

Similarly, 

(^2ui+l,fc+l,<+l  1  .A/n2:E)  =  ^^ol^wol,fc+l,f+l I  ~  ^|Iu>ol,fc+l,t+l  I  —  &|E  1 

(^2u;,fc,t— 1 5 -^n^E)  ~  ^1  |Iu<oO,<:,t— 1 1  —  ^|Iu;oO,<:,/  — 1 1  —  i*lE  | 

{^2u>-i-ljc+l,t  1  I  —  c|E  | 

(b)  trailing  bit  of  w  is  0  and  t  is  odd: 

(^2w,fc  — l,t— li  .A/h^E)  ~  ^2l^'*’o0,fc— 1,1— 1 1  ~  ®  |Iu>oO,fc— 1,<— 1 1  —  O  ^|E^| 
(a:2w+i.it.i,M„iE)  =  =  6-'|Wul  =  t-'|E°| 

{^2w,k—l,ti-^n^E}  ~  ^^al^woO.fc— l,t  I  ~  ^  llwoO.fc— l,f  I  —  ^  |E^j 

(^2u>+l,fc,(+lt -^h^e)  ~  ^sl^u'ol.t.l+l  1  ~  ^  |fwol,fc,t+ll  ~  IE  I 

(c)  trailing  bit  of  w  is  1  and  t  is  even: 

(^2u/,fc  — l,ti  Afn^E)  ~  ^2l^'^o0,fc— l.tl  ~  ®  |Iu(oO,fc— l,tl  —  ®  |E  I 

{^2ijt/+l,k,t—l^  ^ti^e)  ~  1 1  ~  ^  |Iu;ol,fc,t  — 1 1  =  b  ^(E  ( 

(^2w,fc  — l,t+l  1 -^fn^E)  ~  ®3 |IuioO,fc— 1,4+1 1  ~  ^  |Iu;oO,)t— 1,4+1 1  —  b  |E  | 
{^2ui+l,l;,4 '  ~  ®3|Iwol,it,4|  —  ^  |Iu'ol.l;,4|  ~  ^  IE  | 

(d)  trailing  bit  of  u;  is  1  and  t  is  odd: 

(^2ui,fc,4+l  5  A/oIe)  =  ®ol^wo0,fc,4+l  I  —  o|Iu;o0.fc,4+l  I  —  4z|E  | 
{^2u/+l,l:+l,4i -^h^e)  ~  ®ol^<*'ol,^+l,<l  “  ^|Ii»/ol,Jt+l,4|  ~  fcjE  | 

{^2i^,k,tt  ^^n^E)  ~  ®1  |IwoO,fc.4 1  —  ^|Iu)oO,fc,4  i  —  ftjE  | 

(^2u<+1,A:+1,4— 1 1 ~  lIu<ol,fc+I,4— 1 1  ~  c|Iu;ol,it+l,4  — 1 1  —  c|E^| 
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Note  that  in  the  above  enumeration,  we  could  state  precisely  the  possible  nonzero 
inner  products,  and  thus  we  can  precisely  locate  the  positions  of  the  nonzero  entries  of 
More  importantly,  each  inner  product  (involving  two  vectors  in  R^")  was  expressed  as  a 
product  of  an  “ntry  in  M„  and  the  cardinality  of  a  suffix-based  indicia!  set.  In  Chapter  3. 
we  shall  develop  formulas  for  finding  those  cardinalities.  Thus  the  projection  matrix 
can  be  computed  in  time  proportional  to  |S„,/|  and  without  using  any  vectors  in  R^". 

The  analysis  of  the  structure  of  the  column  projection  matrix  P^i  could  be  adapted 
to  the  row'  projection  matrix  P^j  =  Dy^Y’ M^Y  by  considering  directly  how  multiplication 
by  M'  affects  prefix-based  indicial  vectors.  It  is,  however,  more  illuminating  to  exploit  the 
relationship  between  suffix-based  and  prefix-based  indicial  vectors. 

Consider  the  binary  reversal  matrix  R„  6  whose  only  nonzero  entries  are 

ones  in  row  i  and  column  j,  where  the  n-bit  binary  representation  of  i  and  j  are  the  reversal 
of  each  other,  i.e. 


(  Rn)i,j 


1  if  <T„{j)  =  (o„(j))^ 
0  otherwise 


0  <  ij  <  2"  -  1. 


Rn  is  a  reflection:  it  is  symmetric,  involutary  (  Rf,  =  /„)  and  orthogonal ( R^Rn  =  Rj,  =  In)- 
If  Cj  G  R^"  is  the  column  of  the  identity  matrix  then  Rn€j  =  ej  where  we  write 
v{{cT„{j))^)  as  J.  Therefore  for  the  suffix-based  indicial  vector  we  have  RnX^,k,t  = 

k  t  since  R„  does  not  change  the  1-bit  count  and  bit  transition  count  of  an  n-bit  string. 
We  thus  have  a  one-to-one  correspondence  between  suffix-based  indicial  vectors  in  Sn,i  and 
prefix-based  indicial  vectors  in  T„j,  and  R„S„,i  =  {RnX  :  x  G  =  7^,/.  By  a  suitable 
arrangement  of  the  columns  of  Y,  we  can  make  Y  =  R„X .  Then 


Pn.i  =  Dy^Y-M^Y  =  Dj,HX’R-JM:{R„X)  = 


since 

Dy  =  Y‘Y  =  (X-R'JiRnX)  =  X'(R’MX  =  X\X  =  Dx- 

The  sparsity  structure  of  R„MjlR„  is  identical  to  that  of  M„  (the  parameter  values  permute), 
as  Theorem  2.4.1  shows.  We  illustrate  it  for  n  =  4.  Note  that  for  a  matrix  B  G 
BR„  is  a  permutation  of  the  columns  of  B  with  columns  i  and  j  interchanged  if  i  =  J 
(0  <  <  2"  -  1),  while  is  a  permutation  of  the  rows  of  B  arising  from  the  same 

interchange  of  rows.  In  particular,  the  remarks  hold  for  U4R4  and  R^iU^R^)  respectively. 
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mu:r4) 


0000 

0001 

0010 

0011 

0100 

0101 

0110 

0111 

1000 

1001 

1010 

1011 

1100 

1101 

1110 

1111 


shall  determine  where  the  nonzero  entries  of  R„U“R„  are.  It  is  eaisily  verified  that  the  (i,j) 
entry  of  U„  appears  as  the  (J,i)  entry  of  R„U’R„,  or  in  terms  of  strings,  the  (r(/i),  i,’(u;)) 
entry  of  t/„  appears  as  the  (r(w^),i;(^^))  entry  of  R„U‘R„  for/i,ui  6  {0,1}".  In  particular, 
for  u  e  {0, 1}".  the  (u(w),  i’(2u;))  and  (r(a;),  t;{2u;+  1))  entries  of  U„  appear  respectively  as 
the  (u((2w)^),i;(w^))  and  (r({2u;+  l)^),r{u;^))  entries  of  R„U‘R„. 

Consider  the  parameter  Uj.  From  Proposition  2.3.1,  appears  as  the  (u(u;),  v(2u;)) 
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entry  of  U„  where  w  G  {0,  l}"  satisfies  a;{l)  ou;(2)  =  10  and  u{n)  =  1.  Let 

A  =  {w  G  {0,1}"  :^l)ot-'(2)  =  10,  u;(n)  =  1}, 
and  B  =  {a;' G  {0,  l}"  :  u>'(l)  o  u;'(2)  =  01,  a;'(Ti)  =  0}. 

We  claim  that 

{((2uj)^,  LJ^)  :  oj  G  A}  =  +  l):tJ^GB}.  (2.2) 

Note  that  |A|  =  |5|  =  2"“^,  and  that  for  each  set  in  equation  (2.2),  each  pair  in  it  can  arise 
from  only  one  string  in  A  (or  B).  It  thus  suffices  to  show  that  for  u;  G  A,  ((2u;)^,lj^)  = 
(u;',  2u)'  +  1)  for  some  uj'  G  B. 

Let  u;  G  A,  and  u;'  =  {2uj)^.  Then  w(l)  =  1,  lj{2)  =  0,  and  u}(n)  =  1.  In  addition. 

u'  =  [a;(2)o  •  •  •  o u;(n)  o  0]^ 

=  0  ou;(n)  o  .  •  •  o  u>(2), 

and  2u;' +  1  =  uj(n)  ou!(n— 1)  o  •  •  •  o  u;(2)  o  1 

= 

So  ((2u;)^,u;^)  =  (u»',2u;'  +  1),  and  u;'(l)  ou>'(2)  =  01  and  w'(n)  =  0,  i.e.  u'  G  B. 

Thus 

{((2u;)^,  :  a;  G  A}  =  {(w^i  2u;^  +  1)  :  u'  G  B}, 

and  u]  appears  as  the  (u(u;'), v(2u>'  +  1)  entry  of  R„U‘R„  where  w'  G  {0,1}"  satisfies 
u’'(l)ou)'(2)  =  01  and  u;'(n)  =  0.  By  similarly  considering  the  other  fifteen  parameters,  we 
can  characterize  their  positions  in  R„U*R„  in  the  same  manner.  It  then  follows  by  applying 
Proposition  2.3.1  to  RnU‘R„  that 


Applying  the  above  theorem  to  R„M‘R„,  we  have 


and  so  has  the  same  structure  as  P^,. 


2.5  Degenerate  column  and  row  projection  matrices 


For  I  =  0  and  fixed  n,  the  column  and  row  projection  matrices,  P^j  — 
and  =  Dx^X'{RnM*Rn)X  respectively,  are  degenerate  in  that  they  do  not  possess  the 
sparsity  structure  present  when  1  >  1.  Instead,  P^q  and  P^q  have  either  2,  4,  6  or  8  nonzero 
entries  per  column. 

Since  no  suffixes  are  involved  when  /  =  0,  we  index  the  rows  and  columns  of  P^q 
by  the  pair  (/t,  #),  where  le.fc.t  is  a  column  of  A'.  The  entry  in  row  (k',  t')  and  column  {k,  t) 
oiPio  is  then  given  by 

W^e.k'rlr 

Let  E  =  le.fc.j.  Then  E°  =  E^  =  Unlike  the  case  /  >  1,  E°  and  E^  each  do 

not  necessarily  contain  strings  with  the  same  leading  bit  pair  and  the  same  trailing  bit  as 
the  suffixes  are  not  specified.  However,  we  can  partition  them  according  to  the  trailing  bit 
of  strings  in  them; 

E°  =  E§UE$,  E^  =  EjuEj, 


where 


Eg  =  Ig.,,,  Eg  =  I?,,,,  and 


“  ^0,k.f> 


Ei 


Jfc.r 


Eg,  Eg,  El  EJ  are  pairwise  disjoint  by  definition,  and  by  Proposition  2.1.5,  they  each 
contain  strings  with  the  same  leading  bit  pair  and  the  same  trailing  bit.  By  Theorem  2.1.8, 
each  of  2Eg,  2Eg  +  1,  ...,  2Ej  and  2El  +  1  are  suffix-based  indicia!  sets,  and  they  are 
pairwise  disjoint  since: 


(a)  by  Proposition  2.1.6, 

(i)  2Eg.  2Eg  -f  1,  2Eq  and  2Eq  +  1  are  pairwise  disjoint 

(ii)  2Ei.  2Eg  +  1,  2E\  and  2Ei  +  1  are  pairwise  disjoint 


(b)  a  string  fi  from  a  set  in  case  (i)  has  /i(n  —  1)  =  0  whereas  a  string  fi  from  a  set  in  ase 
(ii)  has  /<(7i  -  1 )  =  1. 

Applying  Theorem  2.3.2  to  each  of  Eg,  E®,  Eg  and  Ej,  we  have 
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—  (”^oooa^2Eg  "*■  (^oioa^jE®  ”^oii^2E°+i^ 

+(miooi2El  (”^iio^2EJ  '*'  ”^*”^2Ej  +  l 

for  some  parameters  mooo,  ■■■,  ^ui  of  -^fn-  As  before,  we  note  that  if  Eq  =  0.  then 
X  eO  =  ^oeO+i  =  similarly  for  E^,  Ej  and  Ej.  However,  Eq,  E°,  EJ,  EJ  cannot  all 

be  empty  since  E  =  E§  UE®  uEj  uEj  and  E  is  nonempty.  In  addition,  the  nonzero  vectors 
among  x^eo-  •  •  • '  ^2e1+i  pairwise  disjoint  supports.  So  A/„xe  is  a  linear  combination 
of  either  2,  4.  6  or  8  suffix-based  indicia!  vectors.  The  four  possibilities  can  actually  occur, 
as  the  following  example  show’s: 

eg.  (i)  E  =  l4oo={0000},  Eg  =  {0000},  E$  =  Ej  =  Ei=0. 

(j‘i)  E  =  1^,1  =  {0001,1000},  Eg  =  {1000},  E$  =  {0001},  E^  =  E}  =  0. 
(Hi)  E  =  1^2,3  =  {00101,01001, 10010, 10100}, 

Eg  =  {10010, 10100},  Eg  =  {00101},  Ej  =  0,  E}  =  {OlOOl}. 

(ir)  E  =  I® 32  =  {001110,011100,100011,110001}, 

Eg  =  {OClllO},  Eg  =  {100011},  Ej  =  {011100},  E}  =  {llOOOl}. 

Thus  P^Q  has  either  2,  4,  6  or  8  nonzero  entries  per  column.  Since  has 

the  same  structure  as  Mn,  the  same  conclusion  holds  for  P^o- 

2.6  Action  of  the  transfer  matrix  on  general  indicial  bases 

The  careful  reader  would  no  doubt  question  our  choice  of  indicial  sets.  In  particu¬ 
lar,  the  exclusive  use  of  the  prefix  or  the  suffix  as  one  of  the  three  parameters  in  our  indicia! 
functions  might  seem  overly  restrictive.  In  this  section,  we  introduce  a  more  genera!  class  of 
indicia!  sets,  and  consider  how  the  transfer  matrix  acts  on  the  corresponding  indicia!  bases. 
We  will  then  be  able  to  justify  our  choice  of  approximating  subspaces  in  Section  2.7. 

Let  n>1.0<!:<n,0</<n-l  and  u;i,u^2  G  {0,1}*  with  0  <  |u;i  |  -|-  lu;2|  <  n. 
Recall  that  for  a  string  ^  €  {O,  l}",  k(ji)  and  r(/i)  denote  its  1-bit  count  and  bit  transition 
count  respectively.  We  define  a  genera!  indicia!  set  to  be 

•=  {h  e  {0, 1}"  ;  k(/j)  =  k.  r(fj)  =  t,  U7,  is  a  prefix  of  /i  and  u;2  is  a  suffix  of  ^/} 

and  in  analogous  fashion  to  suffix-based  indicia!  sets,  we  can  define  general  indicia!  vectors 
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^”i,w3,k,t  general  indicia!  bases  VnJij,,  with  0  <  Ij  -t-  h  <  n.  We  will  assume  that  n  is 
a  fixed  integer  >  3,  and  omit  it  in  writing  indicia!  sets  and  vectors. 

It  is  clear  that  suffix-based  and  prefix-based  indicia!  objects  are  special  cases  of 
general  indicia!  objects.  Indeed,  we  have: 

3Jld  Jw.fc.t  ~  ^u),e,k,t^  yui,k,t  ~  ^u>,e,k,ti  ^n./  ~  l^n,/,0- 

In  addition,  since  the  leading  (trailing)  bit  of  a  string  can  be  determined  from  its  trailing 
(leading)  bit  together  with  the  parity  of  u  '.it-transition  count  (cf.  Proposition  2.1.3),  we 
have  for  nonempty  general  indicial  sets  the  following: 

=  Ge,^,k4  if  Iwil  =  1  and  W2  #  £, 
and  if  l‘^2l  =  1  and  w,  7^  e. 


Therefore,  5„,/  =  V„,o,/  =  and  Tn,i  -  V„,/,o  = 

We  note  that  by  the  same  type  of  proof  as  in  Theorem  2.1.8,  we  can  show  that: 

Theorem  2.6.1  Let  E  =  GJJ^  >  3>  Mil  >  2  and  Mi]  -1-  lu;2|  <  n.  Then  2E  = 

2E  +  1  =  for  some  X  xvith  |u;||  =  \u!'^\  = 

Mil  -  1  and  (u;Jl  =  m'l  =  M2I  +  1-  Thus  I2E  and  I2E-H  belong  to  the  basis  V„,|u;i|-i,|u.2H-i- 

Consider  now  the  action  of  M„  on  a  general  indicial  basis  n  >  3,  /i  >  2 

and  l\  +  h  <  n.  As  usual,  we  work  with  the  general  duodiagonal  matrix  U„  instead  of  M„. 


Theorem  2.6.2  Let  n  >  3,  l\  >  2  and  l\  +  l2  <  n.  Then 

C/"„span(V„./,./,)  C  span(V„,j,_i,/,+i ). 

In  particular.  Af„span(  Vn,/,./, )  C  span(  ). 

Proof.  Let  2wi,u;j.fc.<  €  and  E  =  G,^,  Since  |wi|  >  2,  E  contains  strings  with 

the  same  leading  bit  pair  and  the  same  trailing  bit.  Applying  Theorem  2.3.2,  we  have 

=  f'nXE  =  a^'xjE  +  «2'‘^'x2E.H  foF  some  i,  j. 

From  Theorem  2.6.1,  I2E  x^e+i  belong  to  V„./,_i.jj+i.  So  UnX^  €  spaji(V„,/,_i,/j+i ). 

□ 
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The  dual  action  of  M*  is  an  easy  consequence  of  Theorem  2.6.2  once  we  determine 
how  general  indicia!  sets  are  affected  by  binary'  reversal.  It  is  easily  verified  that  for  a 
general  indicia!  set 

=  :  M  e  G^h  ^h 

=  ,u;«  .*:.«• 

Thus  iZn^wi  ~  Rn^n,ti,l2 

Corollary  2.6.3  Let  n  >  2,  I2  >  2  and  h  +  h  ^  n.  Then 

Af;span(V„,/,,/,)  C  span(V„,/,+i,/j_i). 

Proof.  Recall  that  RnM’Rn  is  duodiagonal  (Theorem  2.4.1).  Thus 

C  span(V„,/,_i,/,+i )  by  Theorem  2.6.2, 

and  so 


since  .R“*  =  Rn-  ^3 


A/;;span(V„,/,,/3)  C  .ff„span(V„,i.,_i,;j+i) 
=  span{V„,/,+i,/j_i). 


2.7  Choice  of  indicial  bases  and  the  spectral  invariance  con¬ 
jecture 

Recall  that  5„,/  is  the  set  of  all  nonzero  suffix-based  indicial  vectors  a:"  ^ ,  G  R^" 
with  /-bit  suffix  u),  and  that  T„j  is  the  corresponding  prefix-based  indicial  basis.  In  Sec¬ 
tion  2.2.  for  fixed  n  and  /,  we  chose  span(5„,/)  and  span(7^,/)  as  the  subspaces  for  approxi¬ 
mating  the  top  two  column  eigenvectors  and  the  top  two  row  eigenvectors  of  M„.  We  now 
justify  why  these  choices  are  best  in  a  certain  class  of  subspaces  of  the  same  dimension. 

Recall  from  Section  2.6  that  S„j  —  and  T„j  =  We  w’ill  show  in 

Appendix  A  that  the  general  indicial  bases  V„,2./_i,  ....  all  have  the  same  car¬ 

dinality  (see  Corollary  A. 1.5)  and  so  the  subspaces  span(  V„,,,/+i_,),  i  =  1, 2, ...,/,  all  have 
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the  same  dimension.  We  claim  that  among  these  subspaces,  span(Vn,i./)  =  span(5„,/)  gives 
the  best  approximations  to  the  top  two  column  eigenvectors  of  M„,  and  that  span(V„,/,i) 
=  span(7i,,/)  gives  the  best  approximations  to  the  top  two  row  eigenvectors  of  A/„. 

From  the  theory  of  the  classical  power  method  for  computing  dominant  eigen¬ 
values  and  eigenvectors,  we  know  that  for  any  real  vector  x  and  conformable  matrix  J3, 
Bx  has  a  “richer”  component  in  the  top  column  eigenvector  than  does  i,  and  similarly 
for  the  second  column  eigenvector.  So  for  x  6  span{  i  —  2,3, A/„a- 

has  a  richer  component  in  the  top  column  eigenvector  q\  of  A/„  than  does  x.  But  from 
Theorem  2.6.2,  M„x  G  span(  Vn,,_i,/+2-i)-  Therefore,  the  best  approximation  to  qi  from 
span(  Vn,i-i./+2-i)  would  be  better  than  the  best  approximation  to  q\  from  span(  Vn,i./-(-i-i ), 
i  =  2,3,...,/.  In  particular,  the  best  approximation  to  qi  from  span(V„,i,/)  =  span(5„,/) 
would  be  better  than  any  approximation  to  qj  from  span(V„,,,y+i_,),  i  =  2,3,...,/.  So 
span(5„,/)  contains  the  best  approximations  to  the  top  two  column  eigenvectors  of  Ar,, 
among  span( i  =  1,2,...,/.  A  similar  argument  for  A/*  (and  using  Corol¬ 
lary  2.6.3)  shows  that  span(T„,/)  contains  the  best  approximations  to  the  top  two  row 
eigenvectors  of  A/n  among  span(  Vn,i,/+i_i),  i  =  1, 2, . . . , /. 

What  about  the  approximations  to  the  top  tw’o  eigenvalues  of  Af„?  Would  those 
two  special  subspaces  give  better  approximations  tham  the  other  general  indicial  subspaccs? 
To  answer  these  questions,  let 

PnM-.  =  Z?x‘(A'*A/„A'),  Dx  =  X\X 

where  the  columns  of  A'  are  the  vectors  in  and  let  =  Z?3^\A'*A/*A'). 

Then  =  P^j  and  P«,,,  =  f-«,  as  previously  defined.  We  have  the  following  remarkable 

Theorem  2.7.1  (Spectral  invariance)  The  projection  matrices  .P^2/-i>  •  •  ■ 

<M.  P^^^-  P^.2J-^ . P^JA  the  same  set  of  eigenvalues. 
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Chapter  3 

Combinatorics  of  Indicial 
Subspaces 


3.1  Quantitative  analysis  of  1-bit  suffix-based  indicial  sets 

In  this  section,  we  shall  be  solely  concerned  with  the  quantitative  properties  of 
suffix-based  indicial  sets  of  the  form  I”  ^  j  with  jw)  =  1.  As  usual,  n  will  be  a  fixed  integer 
(and  we  omit  it  in  writing  indicial  sets).  We  first  determine  the  values  of  k  (the  1-bit  count) 
and  t  (the  bit  transition  count)  giving  nonempty  indicial  sets,  and  the  cardinalities  of  those 
sets.  We  then  show  that  the  largest  indicial  set  has  cardinality  w  2"/n7r.  Finally,  we  will 
prove  that  the  number  of  nonempty  indicial  sets  is  2  d-  n(n  —  1)  =  O(n^). 

We  now  begin  a  formal  analysis  that  will  establish  Theorem  3.1.2.  There  is  a 
relation  between  k  and  t  for  strings  ending  in  0. 

Lemma  3.1.1  Consider  nonempty  indicial  sets  Then 

(a)  t  =  0  if  k  =  0, 

(b)  1  <  t  <  min(2l-,2(Ti  -k)-l)  if  l<k<n  —  1. 

Proof.  It  is  obvious  that  for  k  =  0,  the  only  nonempty  indicial  set  is  lo.o.o  =  {0.^0,}. 

n 

Consider  now  a  fixed  k  >  1,  Since  k  >  1  and  any  string  /i  G  lo,k,t  tas  trailing  bit 
0,  there  must  be  at  least  1  bit  transition  within  p,  i.e.  f  >  1.  Suppose  p  6  has  at  most 
as  many  I’s  as  O’s,  i.e.  k  <  n-k.  Then  each  1  in  /x  can  contribute  at  most  2  bit  transitions 
(if  preceded  and  followed  by  O’s),  and  so  t  <  2k.  If  instead  p  has  at  most  as  many  O’s  as 
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I’s,  i.e.  k  >  n  -  k,  then  each  0  in  /i  contributes  at  most  2  bit  transitions,  except  for  the 
trailing  0,  which  contributes  at  most  1  bit  transition.  So  t  <  2(n  —  A:  —  1)  +  1  =  2{r  —  k)  —  1. 
Combining  the  two  cases  yields  1  <  t  <  min(2A:,2{n  -  A-)  —  1)  for  nonempty  indicia!  rets 
lo.fc.ji  A’  >  1.  □ 

The  next  task  is  to  compute  for  1  <  A  <  n  —  1, 1  <  t  <  min{2A,2(n  -  A)  -  1). 

This  will  be  accomplished  by  deriving  an  alternative  characterization  for 

Definition  3.1  Let  n  be  a  nonempty  string.  By  a  block  of  O’s  in  a  string  ft,  we  shall  mean 
a  (nonempty)  sequence  of  O’s  in  p,  which  is  preceded  by  (if  possible)  and  followed  by  (if 
possible)  by  1  ’s.  We  make  an  analogous  definition  for  a  block  of  1  ’s  tn  p. 

eg.  p  =  1001110110  has  3  blocks  of  O’s,  3  blocks  of  I’s,  and  3  -f  3  —  1  =  5  bit 
transitions. 


As  the  above  example  indicates,  the  bit  transition  count  of  a  string  p  is  one  less  than  the 
sum  of  its  number  of  blocks  of  O’s  and  its  number  of  blocks  of  I’s.  We  thus  have  the  following 
alternative  characterization  for  lo.fc.t: 

{p  €  {0,  l}”*  :  K(p)  =  A,  r{p)  =  t,  p{n)  =  0.  and  p  has 

if  t  even 

5  +  1  blocks  of  O’s  and  5  blocks  of  I’s} 

’ 

{pe{0,l}”'  :  k(;i)  =  A,  r(/i)  =  t,  ^(n)  =  0,  and /i  has 

if  t  odd 

blocks  of  O’s  and  Ati  blocks  of  I’s} 


Theorem  3.1.2  The  cardinalities  of  the  nonempty  indicial  sets  ,  are  given  by: 

1^0.0, ol  ~ 

(b)  for  I  <  k  <  n  -  1.  I  <  t  <  mln(2k,2(n  —  k)  -  1), 


A-  1 
t/2  -  1 

A  -  1 
(t-l)/2 


n  —  A  —  1  \ 

t/2  j 

n  —  k  —  1 
(t-l)/2 


if  t  even 


if  t  odd 
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Proof.  Case  (a)  is  trivially  true.  Let  k  and  f  be  as  in  case  (b).  Consider  ju  G  lo.fc.f-  L'sing 
the  above  characterization  of  we  see  that  if  t  is  even,  the  k  I’s  and  n  —  k  O's  of  //  arc 

divided  into  t/2  and  t/2  +  1  blocks  respectively;  and  that  if  t  is  odd,  they  are  each  divided 
into  (t+l)/2  blocks.  From  combinatorics,  we  know  that  the  number  of  ways  to  divide  m  O’s 
(or  m  I’s)  into  q  nonempty  blocks  where  the  ordering  of  the  blocks  is  important  is  given  by 
/  m  —  1  \ 

.  The  result  then  follows  by  applying  the  combinatorial  formula  to  the  division 

V  9-1  / 


of  O’s  and  the  division  of  I’s  of  /r  into  blocks.  □ 

The  cardinalites  of  the  nonempty  indicia!  sets  Ii.fc.t  can  be  derived  by  a  similar 
argument.  We  note,  however,  a  particularly  simple  and  useful  relationship  between  these 
indicial  sets  and  those  of  Theorem  3.1.2. 


Definition  3.2  Let  fi  be  a  nonempty  string.  The  ones  complement  p.  of  p  is  the  string 
obtained  by  reversing  O’s  and  I’s.  If  E  is  a  set  of  nonempty  strings,  we  define  E  :=  {J7  ; 
peE}. 

eg.  0110100  =  1001011. 


Proposition  3.1.3  For  all  n  >  1,  0  <  k  <  n  and  0  <  t  <  n  —  1, 


We  thus  have  a  one-to-one  correspondence  between  the  collection  of  nonempty 
lo.fc.t  and  the  collection  of  nonempty  given  by  ones  complementation,  with  corre¬ 

sponding  indicial  sets  having  the  same  cardinality.  Making  the  transformation  k  —*  n  -  k 
in  Theorem  3.1.2,  we  obtain  the  corresponding  result  for 


Corollary  3.1.4  The  cardinalities  of  the  nonempty  indicial  sets  IJ  ^  are  given  by: 

N  |lT.n,ol  =  1. 

(b)  for  l<k<n-l,l<t<  min(21'  -  l,2(n  —  fr)), 


Kk,\  - 


I'-lW  n-fc-l 

t/2  j  I  t/2  -  1 


fc-1  W  n  -  k  -  I 

(t-  l)/2  j  \  (t-  l)/2 


if  t  even 


ift  odd 
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Having  determined  the  cardinalities  of  the  nonempty  indicia!  sets  Iw.it.t  with  |a;|  = 
1,  we  now  turn  to  finding  the  maximum  cardinality  of  such  a  set.  Because  of  the  one-to- 
one  correspondence  previously  mentioned,  we  need  only  consider  maximizing  among  the 
nonempty  indicia!  sets 

We  first  recall  the  floor  and  ceiling  functions  for  real  numbers.  For  a  real  number 
Q,  [aj  (floor  of  a)  is  the  largest  integer  <  a,  and  fa]  (ceiling  of  a)  is  the  smallest  integer 

>  Q. 

eg.  [3.9J  =  [2.1]  =  3  and  [-6.5]  =  f-7.1]  =  -7. 

We  note  that  [a/2j  -|-  fa/2]  =  a,  [— aj  =  —  fa],  f— a]  =  -  [aj  for  all  real  numbers  a.  In 
addition,  for  a  positive  integer  m, 


The  following  theorem  gives  the  maximum  cardinality  of  an  indicia!  set  Io.fc,«-  It 
then  follows  by  Stirling’s  approximation  that  the  maximum  cardinality  of  a  1-bit  suffix- 
based  indicial  set  is  ss  2" Inir.  The  proof  of  the  theorem  is  tedious,  and  may  be  skipped 
without  any  loss  of  continuity. 


Theorem  3.1.5  Let  n  >  1.  For  n  =  0, 1  (mod  4), 


m^|Ig  ^  J  -  |Io,[n/2j-l,2[n/4j  I  ~ 


[n/2j  -  2 
[n/4j  -  1 


and  for  n  =  2,3  (mod  4), 


f"/2l  \ 
L"/4J  / 


max  11 

k.l 


n 


Ho 


0.  [n/2j,2[n/4j-(-l  ' 


Ln/2J  -  1 
fn/4j 


fn/21  -  1  \ 
fn/4j  ) 


Proof  We  sketch  the  main  ideas  and  leave  the  verification  of  the  details  to  the  reader. 
Define 


mo  =  max  |Iojt/|  and  mj  =  max  |Iojt»|- 

k.t  k,t 

t  even  t  odd 


We  seek  max(mo,  m] ). 
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Consider  first  finding  mo.  By  Theorem  3.1.2, 


mo  = 


max 

1  <  it  <  n  -  1 

1  <  t  <  inin(2ib,2(n  —  it)  —  1),  t  even 


k  -  1 
f/2  -  1 


n  —  k  —  1 

t/2 


max 

1  <  it  <  n  -  1 

0  <  t'  <  inin(fc  —  l,n  —  k  ~  2) 


where  t'  =  t/2  -  1  and  f(k,t')  = 


k  -  1 
t' 


fik,t') 


n  —  k  —  1 


t'+  1 


.  For  a  fixed  k, 


fik,t'+l)  >  f(k,t')  o  +  1  < 


k(n  —  k  —  1) 


(3.2) 


(3.3) 


So  the  value  of  t'  maximizing  f{k,t')  for  fixed  k  is  given  by: 

^  |^fc(n  -  fc  -  1)J 

Similary,  for  a  fixed  t,  we  have  t'  +  l<fc<n  —  t'  —  2  and 

f{k  +  l,t')  >  fik,t')  ^  k  +  l<  +  1. 

So  the  value  of  k  maximizing  f{k,t')  for  fixed  t'  is  given  by: 

By  considering  the  possible  values  of  n  mod  4,  we  can  show  that 

k  =  [n/2j  -  1,  t'  =  [n/4j  -  1 

is  a  simultaneous  solution  to  (3.2)  and  (3.3).  Recalling  the  change  of  variables  t'  =  tjl  —  1, 
we  have 

,,  ,  /  l»/2J  -  2  \  (  f»/2l  \ 

=  W4J-1  j(ln/4J  )■ 

Consider  now  finding  mi.  By  Theorem  3.1.2, 


mi  =  max 

1  <  It  <  n  -  1 

1  <  <  <  inin(2it,2(n  —  it)  —  1),  t  odd 


fc—  1  \  !  n  —  k  —  1 

(t-  l)/2  j  i  (t-  l)/2 


max 

\  <k<n-\ 

0  <  t'  <  inin(it  — l,n-/t  —  1) 
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where  t'  =  {t  -  l)/2  and  g{k,t')  =  Since  g(k,t')  =  g{n  -  k,t'), 

we  conclude  by  symmetry  that  for  a  fixed  t',  the  value  of  k  maximizing  g{k,  t')  is  k  =  [n/2j . 


n  /ol  f  ]f  rn/21-1 

mi  =  max  fl(|n/2J,t)=  max  I  11 

0<t'<Ln/2j-l  0<t'<[n/2j-l  I  f  M 


Since 


[n/4j  =  1^ 


[n/2\  -  1 


for  all  n. 


and  Ln/4j  = 


if  n  =  0,1,2  (mod  4) 
j  if  n  =  3  (mod  4) 


we  conclude  by  formula  (3.1)  that  t'  =  [n/4j  gives  the  maximum  value.  Recalling  the 
change  of  variables  t'  =  {i  —  l)/2,  we  have 


=  |Io.[n/2j,2Ln/4j+l  I  “ 


Ln/2J  -  1  W  rn/21  -  1 


[n/4j 


[n/4j 


We  now  find  max(mo,  tni).  It  is  easily  verified  that 

mi  ^  i[n/2\  -  1)( rn/21  -  in/4j) 
mo  Ln/4j  [n/2] 


and  that 


mi  >  mo  O  r^/21  (L”/2j  -  1)  >  L”/4j  {n  -  1) 


n  =  2,3  (mod  4). 


The  theorem  then  follows.  □ 


Recalling  that  lo.fc.t  =  Ii,n-fc.<i  we  obtain  the  corresponding  result  for  nonempty 
by  making  the  transformation  k  —*  n  -  k\ 


Corollary  3.1.6  Let  n  >  1.  For  n  =  0,1  (mod  4), 


/  r«/2i  \ 

-  |I",r„/2l  +  l,2Ln/4ji  -  j 


[n/2j  -  2 
[n/4j  -  1 
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and  for  n  =  2,2  (mod  4), 


k,t 


1^1,  [n/2l,2Ln/4j  +  l  1 


[71/2]  -  1  \  /  Ln/2J  -  1  \ 
Ln/4J  j  \  [n/4j  ] 


We  now  count  the  number  of  nonempty  indicia!  sets  I  ,,j,,ith  |a;|  =  1.  This  is  easily 
accomplished  using  the  bounds  on  k  and  t  from  Theorem  3.1.2. 


Theorem  3.1.7  The  number  of  nonempty  indicial  sets  with  |a>l  =  1  zs  2  +  n{n  —  1). 


Proof. 


By  Theorem  3.1.2,  the  number  of  nonempty  is: 

n— 1  n— 1 

1  +  ^2  min(2fc,2(Ti  -  A:)  —  1)  =  1  +  ^  (after  some  simplification) 


fc=i 


=  1  + 


fc=i 

n{n  —  1) 


From  the  one-to-one  correspondence  between  nonempty  and  nonempty  Ii.*,-,;.  we  de¬ 
duce  that  the  number  of  nonempty  is  also  1  -f  n(n  -  l)/2,  and  we  have  the  desired 
count.  □ 


3.2  Reduction  principle  for  indicial  sets 

Our  goal  in  this  section  is  to  prove  a  reduction  principle  relating  arbitrary  suffix- 
based  indicial  sets  to  l-bit  suffix-based  indicial  sets.  We  can  then  apply  the  principle  to 
extend  the  results  of  Section  3.1. 

The  reduction  principle  transforms  indicial  sets  IJJ  ^ ,  with  |u;|  =  /  to  1-bit  indicial 
sets  by  dropping  the  last  I  -  1  bits  of  strings  in  Central  to  the  proof  of  the  principle 

are  the  following  fundamental  equalities  for  the  1-bit  count  and  bit  transition  count  of  a 
nonempty  string. 

Proposition  3.2.1  Let  n  >  1  and  p  G  {0,  l}".  Then 

k(p)  =  k(p{1)o  ■  ■ -o  p(n-l+l)) K{p{n-l+2)o  ■  ■ -o  p(n)) 

and  t{p)  =  T{p{l)  o  ■  ■  ■  o  p{n-l+l))  +  T(p{n-l-\-l)  o  ■  ■  ■  o  p{n)). 

We  first  show  that  dropping  the  last  /  -  1  bits  of  strings  in  indeed 

transform  it  into  a  1-bit  indicial  set. 
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Lemma  3.2.2  Let  n  and  I  be  positive  integers,  with  n  >  2  and  2  <  I  <  n,  and  let  u  be  a 
string  of  length  1.  Then 

where 


k'  =  k(ll>(2)  o  ■  ■  •  o  u;(l)) 

and  t'  =  t  —  T(uf). 

Proof.  Let  p  6  ~  /^(l)  o  •  *  •  o  — ^+1)-  Then 

/x(n-/+l)o  ...o;i(n)  =  u, 

k{p')  =  k{p)  -  K{p{n  — I +2)o  ■■■  o  p(n))  by  Proposition  3.2.1 

=  k-k', 

and  t{p')  =  t{p)  —  T{p{n  —  l+l)  o  •  •  •  o  p{n))  by  Proposition  3.2.1 
=  t'. 

Co  ,,r  P  jn-f+I 

Conversely,  let  p'  €  ^Z(ilk-k',t'^  ^  =  M' o [u>(2) o  •  •  .oa;(/)]  =  /i'(l)o  •  ■•op'{n- 

/)  o  u;(l)  o  [u;(2)  o  •  ■  •  o  u;(/)].  Then 

p{n-l+l)o  ..  -o  p(n}  =  w, 

k{p)  =  K(/i')  +  k(w(2) o  •  •  •ou>(/))  by  Proposition  3.2.1 

=  k, 

and  t(p)  =  r(p')  +  o  u{2)  o  ■  ■  •  o  u;(l))  by  Proposition  3.2.1 

=  t. 

So  p  e  j  and  p'  =  p(l)o  •  ■  ■  o  p(n-l+l).  □ 

Theorem  3.2.3  Let  n  and  I  be  positive  integers,  with  n  >  2  and  2  <  I  <  n,  and  let  u>  be 
a  fixed  string  of  length  /.  There  exists  a  one-to-one  correspondence  between  the  collection 
If  of  nonempty  suffix-based  indicial  sets  IJJjt.f  collection  If  of  nonempty  1-bit 

suffix-based  indicial  sets  Furthermore,  corresponding  indicial  sets  have  the  same 

cardinality. 
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Proof.  Define  the  function  f  :ip  -*  by: 

From  Lemma  3.2.2,  =  ^ZiiyLk-r  where 

k'  —  /c(u;(2)  o  •  -  •  ouj{l)) 
and  t'  =  t  —  t{u). 

We  show  that  /  is  a  one-to-one  correspondence  between  I"  and  with  the 

desired  property. 

Suppose  some  €  If.  Then 

Tfi  — _  Yn-i+1 

— r(tj)  w(l),fcj— jk'.tj  — ■r(u») 

and  so  ki  =  fcj  and  ti  =  <2-  Thus  /  is  one-to-one.  Furthermore,  /  is  onto  jJ’-'+i  since  for 

Kmx,  €  IT-'*'.  e  V  “><1  =  Cr.K-  I" 6 1,". 

/(!"  k.t)  same  cardinality  since  for  p.\,H2  € 


because 


/ii(n-/-|-l)o---o|ii(n)  =  /i2(n-l  +  l)  o  •  •  -  o  /i2(”)  =  cJ.  □ 


There  is  a  similar  reduction  principle  relating  general  indicia!  sets  to  suffix- based 
indicia!  sets.  We  refer  the  interested  reader  to  Appendix  A  for  its  statement  and  proof. 


3.3  Quantitative  properties  of  suffix-based  indicial  sets  and 
projection  matrices 

In  this  section,  we  extend  the  results  of  Section  3.1  to  arbitrary  suffix-based  indicial 
sets  using  the  reduction  principle  proved  in  Section  3.2.  We  can  then  deduce  the  order  of  our 
projection  matrices  (or  equivalently,  the  cardinalities  of  our  indicial  bases)  and  the  number 
of  nonzero  entries  in  them.  We  shall  in  fact  show  that  the  indicial  bases  and  Tn.i  both 
have  cardinality 

2-(l+ 


2 
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Before  we  begin  a  formal  analysis,  we  present  some  numerical  results  to  get  a  feel 
for  the  quantitative  behaviour  of  our  indicia!  subspaces.  Table  3.1  gives  the  cardinality  of 
the  indicia!  basis  5„,/  and  the  maximum  size  of  an  indicia!  set  I"  ^  ,  (with  |u;|  =  /)  for  1  —  2,4. 
The  latter  quantity  is  by  definition  equal  to  the  maximum  number  of  ones  appearing  in  a 
basis  vector  in  S„j.  We  see  from  the  table  that  the  indicia!  bases  have  cardinalities  which 
are  very  small  compared  to  2";  in  contrast,  the  basis  vectors  have  a  large  number  of  nonzero 
entries.  For  example,  for  n  =  30  and  /  =  2,  we  approximate  by  the  subspace  spanned 
by  the  1628  basis  vectors  in  530,21  the  “largest”  of  which  has  5.95  x  10®  ones  appearing  in 
it. 


n 

2" 

l*5n,2l 

ui.k.t 

M  =  2 

|5„.4| 

ut,  k,  t 

i“'i  =  4 

5 

32 

28 

2 

32 

1 

10 

1024 

148 

20 

352 

6 

15 

32768 

368 

400 

1072 

120 

20 

1.05  X  10® 

688 

8820 

2192 

2520 

25 

3.36  X  10" 

nos 

2.33  X  10® 

3712 

63504 

30 

1.07  X  10^ 

1628 

5.95  X  10® 

5632 

1.59  X  10® 

Table  3.1;  Combinatorial  properties  of  sufiix-based  indicia!  sets 


We  now  turn  to  extending  the  results  of  Section  3.1.  Let  n  and  I  be  fixed  positive 
integers  with  I  <  n.  From  Theorem  3.2.3,  there  is  a  one-to-one  correspondence  /  between 
the  collection  of  nonempty  suffix-based  indicia!  sets  (for  a  fixed  u;  G  and  the 

collection  of  nonempty  1-bit  suffix-based  indicial  sets  given  by: 


/(I 


n 


^  jn-Z+l 

— k(u»(2)o..-ou»(/)),I  — r(u;)  ’ 


By  making  the  appropriate  substitutions  (n  by  n  -  /  -|-  1,  by  -  k{u>(2)o  ■  ■  -0^(1))  and  f 
by  i  -  t{u!))  in  Theorems  3.1.2  and  3.1.5  and  Corollaries  3.1.4  and  3.1.6  of  Section  3.1.  we 
obtain  corresponding  results  for  nonempty 

Theorem  3.3.1  Let  n  be  a  positive  integer,  and  u;  be  a  fixed  string  of  length  I  <  n.  Let 
k'  =  k(u;(2)  o  •  •  •  o  u;(/)).  The  cardinalities  of  the  nonempty  indicial  sets  I"  jore  given  by: 

(a)  i/u;(l)  =  0; 


(t)  |I 


"  I 

w,A:',r(w)  > 


=  1, 


45 


(it)  fork'  +  l  <k<  n-l  +  k',  r(u;)+l  <t  <  T{u})  +  imTi{2{k  —  k'),2{n  —  l  —  k  +  k')  +  l), 


k-k'  -I 


n  —  I  —  k  +  k' 


(t  -  t{lj))/2  -  1  /\  (<-r(u;))/2 


k-k’-l 


n  —  I  —  k  +  k' 


(t-T{u)-\)l2  /  \  (f-r(a;)-l)/2 


ift  =  r(w)  (mod  2) 


ift=  r(a;)  +  1  (mod  2) 


(b)  t/u;(l)  =  1; 


l^w,n-i+Jt'  +  l,r(w)l  ~ 

(it)  fork'+l  <  k  <  n-l+k',  r(tL))+l  <  <  <  T(a;)+min(2(fc— 2(n— /— fc+fc'+l)). 


k-k'-l 


n  —  I  —  k  +  k' 


it  -  r(a;))/2  \  (t  -  r(u;))/2  -  1 


K,,t\  = 


k-k'-l 


n  -  I  -  k  k' 


(t-r(u;)-l)/2  /  \  (t-r(a>)-l)/2 


ift  =  t{uj)  (mod  2) 


ift  =  t(u>)  +  1  (mod  2) 


Theorem  3.S.2  Let  n  be  a  positive  integer  >  1,  and  u  be  a  fixed  string  of  length  I  <  n. 
Let  k’  =  K(a>(2)  o  •  •  •  o  u>(/)).  For  n  =  I  -  1,1  (mod  4), 


L(n-/+l)/2j-2  W  [(„_/+l)/2l  \ 
L(n-/+1)/4J-1  [(„_/+l)/4j  ; 

|I".L(n-<+l)/2j+/t'-1.2L(n-J+l)/4j  +  r(u,)l  ^ 

l^w,r(n-/+l)/2l+fc'+l,2L{n-/+l)/4j  +  r(w)l  ^ 


and  for  n  =  I  +  1,1  +  2  (mod  4), 


K,t 


[(n-/+l)/2j-l  \  (  [(n-/+l)/2l  -1 


L(n-/+1)/4J 


[(n  -  /  +  l)/4j 


_  1  L(n-i+l)/2j+fc'.2t(n-/+l)/4j+r(w)+ll  t/t>^(l)-0 

I  l^2.r(n-/+l)/2l+fc'.2l(n-i+l)/4j+r(u;)+ll  if  (^(1)  =  1 

Our  next  task  is  to  count  the  number  of  nonempty  sufiix-based  indicia!  sets  lZ.k,t 


with  |u;l  =  /. 
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Theorem  3.3.3  The  number  of  nonempty  indicial  sets  I"  ^ ,  with  pl  =  Z  >  0  ts 


2'(1  + 


(n_-Z  +  l)(n-/))<^^2i-\ 


Proof.  Let  w  be  a  fixed  string  of  length  /.  From  the  proof  of  Theorem  3.1.7,  we  know  that 
there  are  l  +  (n  — Z4-l)(n  — Z)/2  nonempty  indicial  sets  of  the  form  by  Theorem  3.2.3, 

we  have  exactly  that  many  nonempty  indicial  sets  of  the  form  I"  jt  f .  Summing  over  the  2^ 
possible  u’s,  we  obtain  the  desired  count.  □ 

We  apply  the  above  results  to  finding  the  order  and  the  number  of  nonzero  entries 
of  the  column  projection  matrix  P^i  and  of  the  row  projection  matrix  P^i  for  /  >  0.  We 
note  that  these  two  matrices  have  the  same  order  and  the  same  number  of  nonzero  entries 
(see  Section  2.4). 

An  immediate  corollary  to  Theorem  3.3.3  gives  us  the  order  of  the  projection 

matrices: 


Corollary  3.3.4  The  cardinality  of  the  indicial  bases  Snj  and  7^  /,  Z  >  0,  are  both  2^(1  + 
(n  -  Z  +  l)(n  -  Z)/2).  Consequently,  the  projection  matrices  P^j  and  P^j,  I  >  0,  both  have 
order  2*(l  +  (n-Z  +  l)(n  -  Z)/2). 

Our  final  result  concerns  the  number  of  nonzero  entries  in  the  projection  matrices. 


Theorem  3.3.5  The  number  of  nonzero  entries  in  each  of  P^j  and  P^i,  I  >  0,  is 

Proof.  Referring  to  the  analysis  of  the  possible  nonzero  inner  products  occuring  in  the 
construction  of  the  column  projection  matrix  P^j  (see  Section  2.4),  we  see  that  the  number 
of  nonzero  entries  in  Pn.l  is  2(|f'’|  +  |f*|),  where 

=  {nonempty  E°  where  E  =  Iw.fc.j  with  |u>|  =  Z} 
and  =  {nonempty  E^  where  E  =  w'ith  |u;|  =  Z}. 

We  remind  the  reader  that  E°  and  E^  are  subsets  of  strings  in  E  having  2"**  leading  bit  0 
and  1  respectively. 

Again  using  the  fact  (Proposition  2.1.3)  that  the  trailing  bit  and  transition  count 
t  of  a  string  determines  its  leading  bit,  we  see  that  each  E°  £  F”  is  equal  to  some  Gt„,o0.u;,fc,» 
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with  |u>i|  =  1  and  |u)|  =  /,  and  that  each  nonempty  G^joo,ui,fc,t  with  jwil  =  1  and  1^1  =  / 
is  in  Thus  =  {nonempty  G^, oO,u-,it,«  with  |a;i|  =  1  and  |u;|  =  /}•  Similarly,  - 

{nonempty  G^^noi,u;,fc,«  with  (a;ij  =  1  and  (u;|  =  /}.  Therefore 

f  °  U  =  {nonempty  with  |u;i|  =  2  and  |u;|  =  /}. 

Since  n  £  *  =  0,  we  have 

2(|^“|  +  (^M)  = 

=  2  •  Knonempty  with  u;  =  /  +  1}|  by  Theorem  A.1.4 

2'+^  (1  +  by  Theorem  3.3.3.  □ 
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Appendix  A 

Reduction  principle  for  general 
indicial  sets 


In  Section  3.2,  we  proved  a  reduction  principle  (Theorem  3.2.3)  relating  sufiix- 
based  indicial  sets  to  1-bit  suffix-based  indicial  sets.  Our  goal  in  this  appendix  is  to  prove 
a  similar  reduction  principle  relating  general  indicial  sets  (defined  in  Section  2.6)  to  suffix- 
based  indicial  sets. 

VVe  first  introduce  the  shift  operators  4-  and  —  on  bit  strings.  For  a  nonempty 
string  tj  =  u;(l)  o  u;(2)  o  •  •  •  o  tj(|u;|),  we  define 

u> -I- 1  :=  u;(2)  o u;(3)  o  •  •  •  ou;()a;|) o  w(l), 

u;  —  1  :=  u;(|a;|)  o a;(l)  o  •  •  •  oa;(|u;|— 1). 

We  also  define  e-i-l  =  e-  l  =  £.  The  result  of  applying  -|-  i  times  on  a  nonempty  u; 
(1  <  2  <  |wj)  will  be  denoted  by  w  -b  i,  and  that  of  applying  -  i  times  by  u;  —  i.  Thus 

uj  +  i  :=  u;(i-t-l)  oa;(2-)-2)  o  • .  •  o  wd^l)  ou>(l)  o  •  •  •  o  u2(i), 

u)  ~  i  ;=  u;(|u;| -2-f  1)  ou;(|u;l-i-b2)  o  •  •  •  ou)(|u2|)  o  u2(l)  o  •  •  •  o  U2(|a2| -i). 

Note  that 

(w  ±  i)(fc)  =  u;((fc  ±  2  -  1)  mod  |u;| -b  1),  fc  =  1, . . .,  |a;|.  (A.l) 

If  E  C  {0, 1}"*  is  a  set  of  strings  of  length  m,  m  >  2,  we  define  E  -b  2  :=  {w  -b  2  :  u;  e  E} 
and  E  -  2  :=  {u;  -  2  :  u2  e  E}  for  1  <  2  <  m. 

We  note  the  following  properties  of  the  operators  -b  and  - . 
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Proposition  A. 1.1  Let  m  be  an  integer  >  2,  and  let  6  {0,  l}”*,  and  E],E2  C 

{0,1}"*.  Then  for  1  <  i  <  m,  the  following  equivalences  hold: 

=  LJ2  O  LVi  ±  i  =  iJ2  ±  t, 
and  El  =  E^  o  Ei  ±  i  =  E2  ±  i. 

Proposition  A. 1.2  Let  m  be  an  integer  >  2,  and  let  u  €  {0, 1}"*,  and  E  C  {0, 1}"*.  Then 
for  1  <  i  <  m, 

(u>  +  i)  -  i  =  (aj  -  i)  +  i  =  u), 

and  (E  +  i)  -  i  =  (E  —  i)  +  i  =  E. 

We  also  have  the  following  relationships  between  the  1-bit  count  and  the  transition 
count  of  u;,  u  -I-  i  and  uj  —  i: 

k{u  ±  i)  =  (A. 2) 

r{oj  -f  t)  =  r(a>)  -  r(u;(t)  ow(t-fl))  +  r(w(|tjl)  o  w(l)),  (A.3) 

and  r(u)  -  i)  =  t(u;)  -  r(u>(|u>|-t)  ou)(|u;|-t-|-l))  +  r(a;(|u;|)  ou;(l)).  (A.4) 

Formulae  (A.3)  and  (A.4)  arise  from  the  fundamental  equality  for  the  transition  count  of  a 
nonempty  string  p  (cf.  Proposition  3.2.1): 

f(/i)  =  t(p(1)  op(2))  +  t{h(2)  o  n{3))  +  •  •  •  +  T(p(|/i|  - 1)  o /i(|/i|)). 

As  usual,  we  let  n  be  a  fixed  integer  >  3.  By  perfornoing  appropriate  shifts,  we 
can  transform  general  indicia!  sets  to  suffix-based  ones,  and  vice  versa.  This  forms  the  basii. 
of  Theorem  A. 1.4  below. 

Lemma  A. 1.3  Let  li  and  I2  be  positive  integers,  with  2  <  l\  I2  <  n.  Let  ,  be  a 

nonempty  general  indicial  set  with  |wi|  =  /i  and  \u}2\  =  h-  Then 

+  ih  -  1)  = 

where 

u;'  =  u;2  o  wi  ( 1 )  o  • .  •  o  u>i  (/i  -  1 ), 
and  t'  —  t  —  t(ijJi{1^  —  l)ou;i(/i))-f-  r[uj2{l2)  oa>i(l)) 


(A.5) 

(A.6) 
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Proof.  The  equality 

,k,t* 

follows  from  the  fact  that  the  traihng  bit  and  the  parity  of  the  transition  count  of  a  nonempty 
string  determines  its  leading  bit  (Proposition  2.1.3). 

Let  n  6  and  /r'  =  /r  +  (/i  -  1).  By  definition,  fi{l)  o  •  •  -o  //(/j)  =  aJj,  and 

ti(n - I2  +  I)  o  ■■■  o  n(n)  =  LJ2.  ihen 


k(h') 

Tin') 


f.i'in  —  l  +  2)  o  ■ .  •  o  /i'(n) 


and  /i'(l) 


k  by  formula  (A.2), 

r(/x)  -  o  +  r(^(n)  0  /i(l))  by  formula  (A. 3) 

t  -  r(wi(/i  -1)  ou?i(/i))  +  r(u;2(/2)  ou;i(l)) 
t'  by  formula  (A.6), 

[n'(n-li-l2  +  2)  o  •  •  •o/i'(n-/i  +  l)]  o 
(/i'(n-/i  + 2)0  •• -0/(71)] 

(//(n-/2  +  l)o--.o^(Ti)]o{^(l)o---o^(/i-l)] 

UJ2  0  wi  ( 1)  o  tjj  (2)  o  •  •  •ou?i(/i  —  1), 
w'  by  formula  (A.5), 

(|Z+(/,-l))(l) 

^((/i  -  1)  mod  n  +  1)  by  formula  (A.l) 

M/i) 


So  /I  6  Gull (<1  ),w',fc,7'  3nd  (Gull )  +  (/i  —  1)  C  G,^^ ^ 

Conversely, let /i'  G  ,, ,,,  and /x  =  /-(/j-i).  By  definition,  ;i'(l)  =  u;i(/i) 

and  ^'(n-/i-/2  +  2jo---o/x'(n)  =  0^2  ou;i(l)o  •  •  •  ou;i(/i  - 1).  Then 


K(/i)  =  k  by  formula  (A.2), 

t{h)  =  t' -  r{fx'{n  —  li  +  l)  o  fji'(n-lt+2))  +  T{^i'(n)  o  n'(l)) 
by  formula  (A.4) 

=  t'  —  T(u2(h)  owi(l))  -f  T(iOi(li  —  l)  ou;](/])) 

=  /  by  formula  (A.6), 

/x(  1)  o  •  •  •  o  ^(/i)  =  [//'(n-/i +2)  o  -  •  •  c  ^'(ti)]  o /x'(l) 
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=  [a;i(l)o  .••oa;i(/i-l)]ou;i(/i) 

= 

and  /z(n  — /2  +  I)  o  •  • -o //(n)  =  — /i  —  Z2  +  2)  o  •  •  •  o + 1) 

So  and  /x'  =  [fi'  —  (li  —  1)]  +  (/i  —  1)  =  //  +  (/i  —  1).  Thus  j,  C 

Theorem  A. 1.4  Let  I,  /j  and  I2  be  positive  integers,  with  2  <  I  <  n  and  l\  +  I2  =  /■  There 
exists  a  one-to-one  correspondence  between  the  collection  of  nonempty  general  indicial 
sets  with  |u;i|  =  li  and  |w2|  =  h,  collection  Ii_i  of  nonempty  suffix-based 

indicial  sets  lui.fc.t  uiith  |tj|  =  /  —  1.  Furthermore,  corresponding  nonempty  indicial  sets  have 
the  same  cardinality. 


Proof  We  first  consider  the  case  li  =  1.  Recall  that  the  trailing  bit  and  the  transition 
count  of  a  bit  string  determines  its  leading  bit  (Proposition  2.1.3).  So  for  G^^j  t  £  Gij2- 
we  have  for  G  I/j,  we  have  for  some  1-bit 

string  u'.  Thus  =  Ij,,  and  we  have  a  trivial  one-to-one  correspondence. 

Consider  now  the  case  li  >  1.  From  Lemma  A. 1.3,  we  know  that  for  G^|  ,i.,.j,jt.,  £ 

+  (/]  -  1)  £  I;_i. 

Similarly,  we  can  show  that  for  Iu,,fc,f  €  T/_i. 

(Iu;,fc.f)  -  (^1  -  1)  G  Gl,.l2- 

We  can  thus  define  a  function  /  :  ?/,  ./j  — ♦  I/-1  given  by: 

f(G,J^,U2.k.t)  =  (Gu,i,u;j.it.t)  -f  (/l  -  1). 

We  show  that  /  is  a  one-to-one  correspondence  with  the  desired  property.  By 
Proposition  A. 1.1,  f  is  one-to-one.  Furthermore,  f  is  onto  J/_i  since  for  €  Ii-i. 

(W-.»)  -  (/i  -  1)  e  and  /((I,.,u)-  (/i  -  1))  =  ((Iw.fc.*)  -  (h  -  1))+  (/i  -  1)  = 

Finally,  for  G  the  cardinality  of  is  the  same  as  that  of 

since  for  /xi,M2  G 

Ml  7^  =>  Ml  +  {^1  -  1)  M2  +  (ii  -  1) 


by  Proposition  A. 1.1.  □ 
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Corollary  A. 1.5  The  general  indidal  bases  V„,/,i  all  have  the  same 

cardinality. 

We  leave  it  to  the  reader  to  extend  the  results  of  Section  3.3  using  Theorem  A.  1.4. 
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